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ABSTRACT 

A factor analytic study of the Test of English as £ 
Foreign Language (TOEFL) was undertaken to determine the component 
abilities that underlie performance on the test for several major 
language groups: African, Arabic, Chinese (non-Taiwanese), Farsi , 
Germanic, Japanese, and Spanish. Evidence was found that three major 
factors underlie performance on the TOEFL and that these factors are 
relatively unambiguous in their interpretation. A factor underlying 
the listening comprehension section was noted for each language 
group; however, there were differences among the language groups in 
the interpretation of two of the factors. The African, Arabic, 
Chinese, and Japanese groups were generally similar on a factor 
underlying performance on structure, written expression, and reading 
comprehension items; and on another separate factor underlying 
vocabulary items. The Spanish and Germanic groups were also similar 
on each of two other factors, which correspond to the TOEFL subscores 
(structure/written expression, and reading comprehension/vocabulary). 
The vocabulary factor exhibited positive correlations with age and 
degree-intentions in nearly every language group, suggesting that 
vocabulary is the most likely of any of the abilities to develop with 
training or experience. Implications for the interpretation of TOEFL 
subscores are discussed. (SH) 
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The Test of English as a Foreign Language (TOEFL) was developed in 1963 by 
a National Council cn the Testing of English as a Foreign Language, which 
was formed through the cooperative effort of over thirty organizations, public 
and private, that were concerned with testing the English proficiency of non- 
native speakers of the language applying for admission to institutions in the 
United States. In 1965, Educa*'Onal Testing Service (ETS) and the College 
Board assu^^ed joint responsibility for the program and in 1973 a cooperative 
arrangement for the operation of the program was entered into by ETS, the 
College Board, and the Graduate Record Examinations BosTrd. The member- 
ship of the College Board is composed of schools, colleges, school systems, 
and educational associations; Graduate Record Examinations Board mem- 
bers are associated with graduate education. 

ETS administers the TOEFL program under the general direction of a Policy 
Council that was established by, and is affiliated with, the sponsoring 
organizations. Members of the Policy Council represent the College Board 
and the Graduate Record Examinations Board and such institutions and 
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Abstract 



A factor analytic study of the Test of English as a Foreign Language 
(TOEFL) was undertak'^n to determine the component abilities that underly 
performance on the test for several major language groups. Alternative 
analytic procedures were employed that included (a) computing separate 
factor solutions for each language group, (b) rotating separate solutions 
to a common target, and (c) examining the relationships between factors 
and a variety of candidate background variables. 

The structure of the test is discussed in terms of the similarities 
and differences found in each language group. The implications of the 
results for test development, for the interpretation of TOEFL subscores, 
and for English language training are presented. 
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Int roductlon 



The present study was undertaken to provide additional evidence of 
the construct validity of the Test of English as a Foreign Language 
(TOEFL) by determining precisely what component abilities the test 
measures, l«e«, the explanatory constructs that account for examinee 
performance. One general technique for doing so Is factor analysis, a 
set of analytic methods that provides an estimate of the amount and 
nature of test variance that Is attributable to response consistencies as 
these are reflected In Interlteo correlations* 

This factor analytic study of the TOEFL was Intended to add to our 
knowledge of the test* An accumulation o2 knowledge Is considered 
Important, since evidence of a measure's construct validity is generally 
thought to result not from a single study, but from the triangulation 
made possible by a variety of related research (American Psychological 
Association, 1974). Previous studies that have contributed to the 
understanding of what the test measures are those by Angelis, Swinton, 
and Cowell (1979); Angoff and Sharon (1970); Clark (1977); and Pike 
(1979). 

The present study was also intended to provide information about 
minor dimensions of test performance that, although not to be interpreted 
as psychological constructs, might be useful in defining molecular 
dimensions of test-score variance. Such dimensions might be highly 
relevant to sharpening test-content specifications and to guiding 
the construction of additional parallel forms of the test, in much the 
same manner as a previous factor analysis of the GRE Aptitude Test helped 
to define additional relevant item-classification dimensions for that 
test (Powers & Swinton, 1976). 

Finally, since TOEFL is by nature an international test administered 
to an extremely heterogeneous group of examinees, ±\ seemed desirable to 
determine the extent to which the test measures the same constructs for 
V9rious language groups, an assessment that was thought to have important 
Implications for the interpretation of test scores. 

Methods 



Sample Selection 



Samples of examinees ranging in size from approximately 600 to 1,000 
from each of seven major language groups were selected from the total 
population of TOEFL candidates who registered for the November 1976 
International administration. The language groups that were studied 
included African, Arabic, Chinese (non-Taiwanese), Farsi, Germanic, 
Japanese, and Spanish. 



The choice of language groups for analysis was dictated by both 
theoretical and practical considerations. As in the earlier study by 
Angoff and Sharon (1970), the languages were to be (1) as di^^erse as 
possible, representing different branches of the Indo-EuropeaV*airUy and 
several diff^srent non-Indo-European language groups, and (2) repr^.-^ented 
by a sufficiently large number of candidates taking a particular TOEFL 
form, in this case form YTF4. 

The large volume of Chinese- and Fa rsi-s peaking TOEFL candidates 
suggested a aeed to include these two groups. Japanese and Arabic, 
besides representing large volume, provided theoretically interesting 
contrasts with the first two languages. Among African language groups, 
the Guinea-Senegalese gtoup, including Yoruba, Ibo, Efik, and other West 
African languages^ offered a sufficiently large sample for analysis. 
Although the fc^gglutinative Bantu languages were of considerable theoretical 
interest, they were not represented by a sufficient number of candidates 
to be included. Among European languages, Spanish represented the 
largest candidate volume, and Germanic languages (German, Dutch, Swedish, 
Norwegian, Danish, and Icelandic), although relatively low in volume, 
were theoretically important because of their close linguistic affinity 
to English. It was recognized that considerable variety exists within 
the Guinea-Senegalese and Germanic language families, but the relatively 
small number of candidates in each language made pooling necessary. 
Thus, the resulting set of languages contained five of the six examined 
by Angoff and Sharon (1970). We substituted Farsi for Gujarati among 
Indo-European languages because of changes in candidate volume and added 
the Guinea-Senegalese language group to the non-Indo-European set. The 
language groups and samples were identical to those used by Alderman 
(1980) in a study of item performance across language groups. Thus 
comparisons with that study are possible. 



Analyses 

Each of the 149* items in the TOEFL was used as a variable in 
several factor analyses. For each language group, the matrix of tetra- 
choric intetltem correlations was analyzed. Items passed by 9G% or more 
of a group were omitted in order to avoid convergence problems with the 
tetrachoric computation algorithm. Initial correlation estimates were 
based on a method devised by Tucker* and wer ^sed as starting values for 
minres solutions (Harman, 1976). Communal^ estimates were obtained as 
a byproduct of the iterative mimes procedure. By examining the break 
in the magnitude of eigenvaJues and subsequently rotating several numbers 
of factors, according to the varimax criterion we were able to choose a 



One structure item, number 13 in Section II, was not included in the 
operational scoring, because of British/American vocabulary differences, 
and was correspondingly omitted from this analysis. 
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provisional orthogonal minres solution for each language group. On the 
basis of the variations in structure suggested by each group's solution, 
we Specified a common target matrix for all groups* We then performed a 
rotation to this target for each group to insure optimal comparability 
while placing minimal constraints on the emergence of the kinds of 
structural differences that had emerged from the within-group varimax 
rotations. 

When simple structures had been estimated by the above method, 
candidate background variables were regressed on factors through factor 
extension analysis (Dwyer, 1937) in order to determine the relationship 
of each factor to various candidate background characteristics (e«g«, 
sex, age, degree planned, intended major field, number of times TOEFL 
taken, nfitive country, and reason for taking TOEFL). This technique 
provided further interpretive information, alloid.ng examination of the 
relationship of these extension variables to the factors underlying test 
performance, without affecting the factor structure determined for the 
test. 



Results 



Description of Language Groups 

Table 1 shows the number in each language group on which the reported 
analyses are based, and the average number of items answered correctly. 
As is clear, there is substantial variation among the language groups. 
For example Germanic speakers received higher scores than any other group 
on each subtest; Farsi speakers received lower scores than any other 
group. Furthermore, the subtest profiles are different for the groups: 
group performances are not necessarily uniformly high or low on all 
subtests. For example, although African speakers rank second o^Structure 
and Written Expression, they rank only sixth on Listening Coi^ehenslon. 
These between-group differences in the level of English proficiency have 
implications for the interpretation of factors and the comparison of 
structures across language groups. Variation of item difficulty across 
language groups within a subtest is prima facie evidence of multidimension- 
ality of that subtest. Alderman's (1980) study of item difficulties 
reveals that such departure from unidim^msionality exists for these 
samples and for this form of the TOEFL. 



Preliminary Analyses 

Inspection of eigenvalues suggested that different numbers of 
factors might be appropriate for each of the language groups. With the 
exception of the Germanic group, three, four, or five factors appeared 
appropriate for each language group. Trial rotations of the^e numbers of 
factors were carried out. For the Germanic group, however, the size of 
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Table 1 

Mean Number of Items Correct and Standard Deviations for 
Seven Language Groups for Each TOEFL Subtest 



Language 
Group 



N 



TOEFL Scores 



Reading 
Comprehension 
Listening Structure and and 

Comprehension Written Expression Vocabulary 



African 



74 0* Mean 
S.D. 



29.5 
8.5 



28.8 
6.1 



41.2 
9.2 



Arabic 



686 



Mean 
S.D. 



32.4 
9.6 



23.0 
6.9 



35.0 
10.5 



a»inese 998 Mean 

S.D. 



31.5 
8.9 



24.8 
6.4 



37.3 
9.9 



Far si 



987 



Mean 
S.D. 



27.6 
9.8 



19.4 
6.5 



29.6 
9.3 



Germanic 596* Mtan 

S.D. 



43.6 
5.5 



30.7 
1.2 



48.0 
7.1 



Japanese 997 Mean 

S.D. 



30.6 
8.6 



22.8 
6.3 



36.3 
10.1 



Spanish 991 Mean 

S.D. 



35.5 
9.8 



24.7 
7.3 



44.8 
9.0 



*One composition of the combined language groups was as follows: 



Language n 

African 

Efik 68 

Ibo A 13 

Fanti ^0 

Yoroba 219 

Germanic 

Danish 39 

German 314 

Dutch 157 

Swedish 86 

ERIC ' ' 
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elgenvalues suggested that as many as eight factors might be important 
for a good fit to the Item intercorrelations or a comprehensive description 
of the group's test performance. This greater complexity for the Germanic 
group suggests that TOEFL is tapping a larger set of differentiated 
abilities in this group. One reason for this result may be that, because 
of the overall high level of English proficiency ancoantered in the 
Germanic speakers, their responses included less noise attributable to 
guessing, making it possible for a larger number of minor dimensions of 
covariation to appear consistently among items (but see the Discussion 
[p. 13] for a more substantive hypothesis). 

Table 2 gives the numbers of items from each subsection with highest 
loadings on each factor. These are the results of the separate varimax 
solutions for each language group. Those cases in which one-half or more 
of the items of a given type load on a single factor are asterisked. 

Inspection of Table 2 reveals that the great majority of Listening 
Comprehension items load on a single factor in each language group. 
Furthermore, this factor ^ojaa relatively few loadings from other item 
types in most language groups, suggesting that in general, the Listening 
Comprehension items appear to determine a distinct factor, as would be 
expected from their unique aural, rather than written, mode of presentation. 

The Structure and Written Expression grouping of items tend to load 
on a common factor for the African, Chinese, and Japanese groups; but 
for the Arabic, Farsi, and Spanish groups. Written expression determines 
a relatively independent factor. For the Chinese and Total groups — and 
to a lesser extent, for the African, Germanic, and Japanese — Written 
Expression clusters with Reading Comprehension. 

The other operational grouping. Vocabulary and Reading Comprehension, 
forms an empirical cluster for Arabic and Spanish speakers. It also 
exhibits a moderate tendency to cluster for the African and Farsi groups 
and a weaker one for the Germanic speakers in these independent varimax 
rotations. Vocabulary determines a relatively independent factor for the 
Chinese, Japanese, and Total Groups, however, and exhibits some tendency 
to split from Reading Comprehension in the African-, Germanic-, nnd 
Far si-speaking samples. 

To summarize the varimax rotations, we found that Listening Comprehen- 
sion appears to be a univocal factor in all groups, although Structure 
items tend to load somewhat on this factor in the Spanish, Germanic, 
Farsi, and Arabic groups. Structure and Written Expression items form a 
clear cluster for the African, Chinese, and Japanese groups, but Written 
Expression separates from Structure to form a distinct factor in the 
Arabic and Farsi groups. Vocabulary and Reading" Comprehension form a 
relatively clear cluster in the Arabic and Spanish groups, and a weaker 
cluster in the Farsi and African groups. However, Vocabulary splits off 
from Reading Comprehension to form a clearly separate factor in the Chinese, 
Japanese, and Total groups and shows some tendency to split off from Reading 
Comprehension in the African, Arabic, Farsi, and Germanic groups. 
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Table 2 

Number of Items with Highest Loading on 
» Each Factor/Four-factor Varlmax Rotation 



Factor 

Language 







n 


I 




II 






111 


IV 


African 


Listenins Como* 


50 


45* 




2 






2 


1 




Structure 


14(2) # 


1 




5 






7* 


1 




Written Expres. 


25(lj 


0 




7 






18* 


0 




Vocabulary 


30(1) 


4 




22* 






3 


1 




Reading Comp. 


30(1) 


3 




10 






12 


5 




Variance 




12.5 




11.3 






10.8 


3.6 




Roots 29.8 


3.9 3.5 


2.6 2.1 


2.0 


2.U 


1 


.8 


1.7 1.6 


1.6 




Trace 53.38 


U 












Tl,2 - 

in = 


Jo. 14 




Listmlnff Comn. 


50 


44* 




3 






1 


2 




Structure 


14 


5 




6 






1 


2 




Written Expres. 


25 


1 




9 






4 


11 




Vocabulary 


30 


2 




16* 






12 


0 




Reading Comp. 


30 


1 




21* 






6 


2 




Var ianc e 




16.8 




14.7 






7.9 


6.7 




Roots 36.3 


4.2 3.4 


2.3 2.1 


1.9 


1.8 


1 


.7 


1.7 1.6 






Trarp 57 13 














In = 


46.02 






50 


8 




37* 






4 


1 




Structure 


14 


8* 




1 






4 


1 




Written Expres. 


25 


16* 




1 






6 


2 




Vocabulary 


30 


7 




0 






23* 


0 




Reading Comp. 


30 


26.* 




0 






2 


2 




Variance 




17.2 




13.1 






8.6 


3.0 




Roots 33.0 


3.9 3.4 


1.9 1.6 


1.5 


1.4 


1 


.3 








Trace A8.72 














9 

Lh = 


42.18 


Far si 


Listening Comp. 


50 


44* 




3 






• 

2 


1 




Structure 


14 


5 




3 






4 


2 




Written Expres. 


25 


3 




5 






5 


12 




Vocabulary 


30 


3 




10 






12 


5 




Reading Comp. 


30 


1 




24* 






3 


2 




Variance 




15.0 




11.0 






6.5 


6.0 




Roots 29.3 


4.1 3.5 


1.8 1.6 


1.5 


1.4 


1 


.3 


1.3 1.2 





Trace 47.74 



38.42 
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Tabli 2 (Cont'd.) 



Factor 

Language 



Groups 




n 


I 


11 


III 


IV 


German ic 


Listening Comp. 


50(6) 


37* 


13 


0 


0 




Structure 


1A(2) 


6 


3 


5 


0 




Written Expres. 


25(3) 


2 


10 


10 


3 




Vocabulary 


30(5) 


9 


7 


12 


1 




Reading Comp. 


30(8) 




23* 


5 


1 




Variance 






12.3 


9.0 


3. 




Roots 33.5 


6.5 A.O 


3.2 3.0 


2.7 2.5 








Trace 56.75 








• rh2 = 


36.30 



Japanese 



Spanish 



Total 



Listening Comp. 




50 








•J 
J 




o 


o 


Structure 








1 




7* 




o 


4 






25 




0 




18"* 




2 


5 


Vocabulary 




30 




3 




0 


• 


23* 


, 4 


Reading Comp. 




30(1) 




5 




13 




6 


' 6 


Variance 








13.6 




11.2 




9.6 


5.3 


Roots t 31.8 


4 


.2 2.3 


1 


.7 1.6 


1.5 


1.5 


1.4 






Trace 47.15 


















39.63 


Listening Comp. 




50 




45* 




4 




1 


0 


Struccure 




a4 




7 * 




4 




2 


1 


Written Expres. 




25 




6 




15* 




1 


3 


Vocabulary 




30(6) 




2 




8 




16* 


4 


Reading Comp. 




30(2) 




1 




2 




25* 


2 


Variance 








30.3 




13.3 




12.2 


; 8 


Roofs 43.6 


4 


.7.1 3.3 


1 


.9 1.7 


1.6 


1.5 


1.4 






Trace 60.49 


















50.50 


Listening Comp. 




50 • 




7 




34 * 




6 


3 


Structure 




14 




3 




1 

± 




6 


3 


Written Expres. 




25 




19* 




0 




5 


1 


Vocabulary 




30 




4 




3 




19* 


4 


Reading Comp. 




30 




26* 




1 




3 


0 


Variance, 








16.5 




16.1 




1.3 


7.6 


Roots 42.7 


4 


.8 3.0 


2 


.6 1.7 


1.6 


1.4 


1.4 







Trace 59.73 ^ 53^00 



*Ast^risks represent cases in which half or mote of the items of a given type 
have their highest loadings on the indicated factor. 

//Numbers of items in parentheses were passed by 96 percent pr more of subjects. 
They were not used to determine factor structure, but loadings for them were 
obtained t trough extension analysis. 

//#The trace given here is 149 X (the average Tucker communality estimate). 
Thus j;fi2/Trace estimates the proportion of commonr. variance accounted for by the 
O four factors* 
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Thus , although Listening Comprehension items form a separate dimension • 
in all grpups, Written Expression clusters with Structure for ^me native 
languages, but does not for others, and Vocabulary forms a separate 
dimension from Reading Comprehension in most groups. In the cases in 
which Vocabulary forms the most unequivocally independent factor (African, 
Chinese, and Japanese), Structure, Written Expression, and Reading 
Comprehension tend to load on a single factor. In no case do both 
Structure and Vocabulary items display half or more of their loadings on 
the same factor. This last observation allows us to perform further 
Procrustes rotations on th^se minres factor solutions to maximize com- 
parability without distorting the empirical factor structure in any 
group. 



Rotation to an A Priori Target 

The four-factor minres solutions for each language group were 
subjected to an orthogonal Procrustes rotation (Browne & Kristof, 1969]* 
to force them to fit the design structure of the test, to the extent that 
the data wou^d allow, and to facilitate comparisons among the subject 
groups. The target structure consisted of three constrained factors and 
a fourth unconstrained factor. The first target factor was specifie'" as 
maximum loadings on 'A items of middle difficulty from the Listening 
Comprehension section of the test, with loadings on other items unspecified. 

The second target factor was specified as maximum loadings on all lA 
Structure items from Section II of the test, with loadings on other items 
unspecified. The third target factor consisted cf 14 Vocabulary items of 
middle "difficulty ^rom Section III of the test. 

Our rationale for performing these rotations was, first, that we 
wished to facilitate comparisons by orienting the factors of p^'ch group 
as closely as possible, within the constraints of orthogonality, to the 
specifications of the test, and, second, that we wished to allow departures 
from the specified .structure to reveal -themselves uniquely for each 
group. The number of items determining a target factor was limited to lA 
because the structure section of this test form' contained only lA scored 
items. If more than lA items from another section had been specified as 
the target for another factor, the rotation would have tended to obtain a 
spuriously better fit to the factor having more target items. 

If all Listening Comprehension items do indeed form a single factor, 
the remaining nontarget. 36 listening items would be expected to load on 
Factor I Just as strongly as the target items do. The tarjget items 
themselves should, of course, all load more strongly on Factor I than on 
the other three factors. On the other hand, if the Listening Compre- 
hension section taps more than one dimension of performance, no orthogonal 
rotation will force the set of items to fit a single factor well. 
Similarly, if Structure and Written Expression items form a single 
dimension, the Written Expression items should coalesce with the Structure 
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items to fit target Factor II. That Written Expression Items were not 
specified as part of the target matrix, however, makes It possible for 
this expectation to be d Is confirmed by the data. Although a weighting 
procedure could have been employed (e.g. weighting Listening Comprehension 
Items each by 14/ 50), this would not have made possible the Internal 
test of homogeneity afforded by assigning a weight of 1 to 14 Items and a 
weight of "zero" to the remaining 36 Listening Comprehension Items. 

The same rationale applies to Section III of the test. Only 14 
Vocabulary Items are specified In the target. If performance on Section 
III Is In fact unldlmenslonal, the other 16 nontarget Vocabulary Item^ 
and all 30 Reading Comprehension Items should load on Factor III. The 
unspecified fourth factor allows room for Items to split off from the 
specified structure If the data force such splitting. In fact, the 
fourth factor did not prove necessary In describing the structure of the 
test for any language group except the Germanic. 

Interpretation of Target Factors 

The results of the four-factor orthogonal rotations to target for each 
language group are shown In Appendix A, In which all loadings greater 
than .20 are given.. Three summaries of those data will be given here: 

1. the number of various TOEFL Item types having the 
highest loading on each factor for each language 
group (Table 3); 

2. the number of various TOEFL Item types having high 
loadings (.30 or higher) on each factor for each 
language group (Table 4); and 

3. a description of the particular Individual item that 
had the highest loading of any item on each factor 
for each language group (Table 5). 

Although each summary in Tables 3-5 is Important, each is given lii^ 
the order of Importance to the interpretation of factors. That is, ^able 
3 deserves more attention than either Table 4 or Table 5, and so on. 

Factor I, targeted on medium difficulty Listening Comprehension 
items, is clearly defined for each language group by a preponderance 
of high loadings from Listening Comprehension items. Depending on the 
particular language group, from 76 percent to 88 percent of Listening 
Comprehension items loaded higher on Factor I than on any other. In 
addition, from 76 percent to 90 percent of all Listening Comprehension 
items had loadings greater than .30. Finally, for each language group, 
the item having the highest loading appeared in the Listening Comprehension 
section of the test. Factor I, therefore, is labeled Listening Comprehen- 
sion; Factor II, targeted on all Structure items, is less easily character- 
ized than Factor I because of high loadings from several item types and 
because of differences from group to giroup. Factor II is most similar 
for the African, Arabic, and Japanese groups: all or nearly all Written 
Expression items and a majority of both Structure and Reading Comprehension 
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Table 3 

Numbers of Various T3rpes of TOEFL Items Loading Highest 
on Each of Four Factors for Seven Language Groups/Target Factors 



Language 
G roup 

r 



Item 

T yp e.. 



Number 

of 
Items 



Factors 



II 



III 



IV 



Listening Comprehension 
Structure 
African Written Expression 

Vocabulary 

Reading Comprehension 

Listening Comprehension 
Structure 
Arabic '^J^ritten Expression 

Vocabulary 

Reading Comprehension 



Chinese 



Farsi 



Listening Comprehension 
Structure 

Written Expression 
Vocabulary 

Reading Comprehension 

* 

Listening Comprehension 
Structure 

Written Expression 
Vocabulary 

Reading Comprehension 



Listening Comprehension 
Structure 
Germanic Written Expression 

Vocabulary 

Reading Conqjrehension 

Listening Comprehension 
Structure 
Japanese Written Expression 

Vocabulary 

Reading Comprehension 

Listening Comprehension 
Structure 
Spanish Written Expression 

Vocabulary 

Reading Comprehension 



50 
14 
25 
30 
30 

50 
14 
25 
30 
30 

50 
14 

25 
30 
30 

50 
14 
25 
30 
30 

50 
14 
25 
30 
30 

50 
14 
25 
30 
30 

50 
14 
25 
30 
30 



44 
0 
0 
1 
1 

38 
5 
0 
2 
1 

42 
1 

2 
0 
0 

39 

4- 
6 
5 
8 

43 
1 
1 
1 
4 

43 
0 
1 
2 
3 

38 
3 
1 
1 
1 



4 
9 

25 
7 

22 

12 
8 

25 
9 

20 

6 
10 
15 

8 
27 

10 
5 
8 

11 
1 

6 
8 
16 
11 
4 

5 
10 
21 

4 
22 

12 
7 

21 
9 
2 



. 1 
5 
0 

20 
6 

0 
1 
0 
17 
8 

1 
3 
8 
22 
2 

0 
5 
10 
13 
20 

1 
4 
8 
15 
18 

2 
1 
2 
23 
5 

0 
4 
3 
19 
27 



1 

0 

2 
1 

0 
0 
0 
2 
1 

1 
0 
0 
0 

1 
1 

0 

1 
1 
1 

0 

1 

0 
3 
4 

0 
3 
1 
1 
0 

0 
0 
0 

1 

0 



1 
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Tabie 4 

Numbers of Various Types of TOEFL Items Having Loadings Greater 
than .30 on Each of Four Factors for Seven Language Groups/Target Factors 



Language 
Group 


Item 
Type 


Number 

of 
Items 


T 
X 


Factors 
II ill 


TV 


• 


Listening Comprehension 


50 


38 


4 


1 


0 




Structure 


14 


2 


q 


5 


1 


African 


Written Expression 


25 


0 


21 


2 


1 




Vocabulary 


30 


2 


7 


17 


0 




Reading Comprehension 




A - 


17 


5 


A 
u 




Listening Comprehension 


50 


42 


26 


2 


0 




Structure 


14 


6 


A 
u 


3 


1 


Arabic 


Written Expression 


25 


4 


24 


5 


2 




Vocabulary 


30 


4 


8 


19 


3 




Reading Comprehension 




A 


21 


16 






Listening Comprehension 


50 


39 


12 


0 


0 




Structure 


14 


3 


R 


5 


0 


Chinese 


Written Expression 


25 


3 


14 


4 


3 




Vocabulary 


30 


1 


11 


17 


0 




Reading Comprehension 




X 


25 


4 






Listening Comprehension 


50 


40 


24 


3 


1 




Structure 


14 


6 




3 


1 


Far si 


Written Expression 


25 


6 


10 


7 


2 




Vocabulary 


30 


1 


9 


11 


0 




Reading Comprehension 




1 A 


3 


20 


n 




Listering Comprehension 


50 


42 


13 


5 


2 




Structure 


14 


2 


10 


4 


3 


Germanic 


Written Expression 


25 


2 


18 


11 


1 




Vocabulary 


30 


1 


14 


10 


3 




Reading Comprehension 


30 


12 


13 


20 


9 




Listening Comprehension 


50 


40 


8 


4 


1 




Structure 


14 


3 


10 


2 


1 


Japanese 


Written Expression 


25 


1 


19 


4 


2 




Vocabulary 


30 ■ 


2 


5 


17 


0 


• 


Reading Comprehension 


30 


7 


24 


17 


0 




Listening Comprehension 


50 


45 


32 


7 


0 




Structure 


14 


6 


9 


5 


0 


Spanish 


Written Expression 


25 


6 


21 


5 


1 




Vocabulary 


30 


6 


9 


15 


2 




Reading Comprehension 


30 


12 


8 


28 


0 
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Table 5 

Characteristics of Item Loading Highest on 
Each Factor for Each Language Group/Target Factors 



Factor 



Language Group 



African 

Arabic 

Chinese 

Far si 

Germanic 

Japanese 

Spanish 



Size of 
Loading 



.56 
.68 
.61 
.68 
.66 
.63 
.68 



Item 
Number 



LC-11 
LC-31 
LC-32 
LC-27 
LC-48 
LC-25 
LC-27 



II 



African 

Arabic 

Chinese 

Farsl 

Germanic 

Japanese 

Spanish 



.82 
.64 

.(.'♦ 
.51 
.80 
.59 
.62 



WE-29 

WE-16,17,33 

RC-52 

S-10 

WE-22 

RC-4A 

V-18 



III 



IV 



African 

Arabic 

Chinese 

Farsl 

Germanic 

Japanese 

Spanish 

African 

Arabic 

Chinese 

Farsl 

Germanic 

Japanese 

Spanish 



.67 
.60 
.58 
.55 
.71 
.60 
.59 

.44 

.25 
.55 
.36 
.50 
.40 
.40 



V-30 

RC-52 

V-28 

RC-59 

RC-59 

V-25 

V-25,RC-5 

WE-16 

RC-46 

RC-35 

S-8 

V-4 

S-8 

WE-16 
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Items loaded higher on this factor than on any other. The Chinese group 
also had a similar pattern of loadings, but with a smaller majority (60 
percent) of Written Expression Items loading highest on this factor. The 
pattern of loadings greater than #30 Is also similar for this factor 
among the groups mentioned. In addition, the Items loading highest on 
this factor for both the African and Arabic groups are Written Expression 
Items, but are Reading Comprehension Items for the Chinese and Japanese 
groups. Thus, for the African, Arabic, Chinese, and Japanese groups, 
Factor II seems to underlie three different Item types: two dealing with 
writing (Structure and Written Expression) and one dealing with reading 
(Reading Comprehension). 

A pattern similar to that described above was noted for Factor II 
for the Spanish and Germanic groups, with one Important difference: 
Reading Comprehension Items did not load particularly strongly on Factor 
II, but Instead tended to load on Factor III, the Vocabulary target 
factor, thus fitting the soeclf Icatlons of tha three-part TOEFL well. 
For the Farsl group, Reading Comprehension Items also loaded mainly on 
Factor III, instead of on Factor II, and neither Structure nor Written 
ExpiTesslon Items loaded as strongly on Factor II as these Items did for 
other language groups. These three language groups constitute the 
Indo-European languages In this sample. 

Factor 'ill is defined and targeted by high loadings from Vocabulary 
Items for the African, Arabic, Chinese, and Japanese groups. For the 
Germanic and Spanish groups. Factor III Is also defined by high loadings 
from a majority of Vocabulary ^tems, but even more strongly characterized 
by Readlng«Comprehenslon Items, which were carried along In the rotation 
to the vocabulary target. For the Farsl group. Factor III Is defined 
by both Reading Comprehension and Vocabulary Items and also to a lesser 
extent by written expression Items. 

Factor IV Is not readily linked to any particular Item type(s) for 
most language groups. This "residual factor" was carried In the analysis 
In order to give any Interpretable orthogonal residual variance the 
chance to emerge as a factor. For the African language group, however, 
Factor IV Is quite clearly Interpretable as reading speed, while for 
the Germanic group, this factor contrasts Reading Comprehension and 
Vocabulary Items. A summary of factors for the Individual varlmax 
solutions and the solutions rotated to target te given In Table 6. 



Discussion 

Integrating Findings with Other Research 

The present study offers some evidence that (a) three major factors 
underlie performance on the TOEFL and (b) these factors are relatively 
unambiguous In their Interpretation. A factor underlying the Listening 
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Table 6 



Summary of Factors for Several Language Groups 
for Two Methods of Analysis 



Method of Analysis 

Ind iv idual Varlmax Solutions Individual Solutions Rotated to a Target 



Language Group 


Factor 
I 


Factor 
11 


Factor 
111 


Factor 
IV 


Fac t or 

1 


l?o 

11 


S <lv> WML 

111 


Factor 
IV 


African 


LC 


V 


WE,S,RC 


— 


LC 


WE.RC.S 


V 


— 


Arabic 


LC 


RC,V,S 




WE 


LC 


WE,RC,S 


V 




Chinese 


RC.WE.S 


LC 


V 




LC 


RC,S,WE 


V 




Far si 


LC 


RC 


V 


WE 


LC 


(S) 


RC,V,WE,(S) 




Germanic 


LC 


RC,(WE) 


V,(WE) 




LC 


WE,S 


RC,V 




Japanese 


LC 


WE.S.RC 


V 




LC 


WE.RC.S 


V 




Spanish 


LC 


WE 


RC,V 




LC 


WE,S 


RC,V 





KEY: LC - Listening Comprehension 
S - English Structure 
WE ■ Written Expression 

V ■ Vocabulary 
RC ^ Reading Comprehension 



Note > Each item t^-pe is listed under the factor on which the item type has Its greatest number of highest 
loadings, in order of the types having the highest percentage of highest loadings. 

Types In parentheses indicate that the item type also had an equal number of highest loadings on 
another factor. 



Comprehension section was noted for each language group* However, there 

are differences among the language groups in the inti^rptetation of two 

of the factors « Foi the two Indo-European language groups whose languages 

are most Similar to English, e*g*, the Germanic and the Spanish groups, 

the seconJ and third factors correspond with the TOEFL subscores (Structure + 

Written Expression and Reading Comprehension + Vocabulary) that are now 

reported. For most other groups (African, Arabic, Chinese, and Japanese) 

the Reading Comprehension items tend to load on the same factor with 

Structure and Written Expression, with Vocabulary splitting off from 

Reading Comprehension to define a third factor* For Farsi speakers, who 

in this sample are the least proficient of any group with respect to 

total TOEFL scores, the factor structure is the least differentiated of 

any group structure* With the exception of the Listening Comprehension 

factor, each of the other factors is less clearly interpretable* Items 

from each section are more likely to load on several factors, and 

each factor is more likely to have high loadings from several item types 

for the Farsi group than for any other language group. Thus the Farsi 

results are more suggestive of a single Listening Comprehension factor 

and a global factor underlying performance on Sections II and III than 

are the structures of the other groups* 

It is particularly interesting to mention here the results of a 
study by Oiler (1976) that was designed to test the extent to which 
language proficiency is divisible into -separately assessable components* 
Oiler (1979) reports that a factor analysis of TOEFL data for 159 Iranian 
adults in Tehran supported the indivisibility hypothesis of a single 
global factor. Another relevant study, conducted by Hinofotis (1976) and 
reported by Oiler and Hinofotis (l^^iO), in which data were obtained at 
the Center for English as a Second Language, was somewhat more supportive 
of the possibility of separate language skills, particularly speaking 
skills. The point to be made here is that the results of correlational 
studies of language skills may be quite dependent on the particular 
sample used, especially with respect to their overall level of language 
proficiency. One hypothesis that could be investigated is the extent to 
which separate factors (or components of variation) are more likely to 
emerge as the overall language proficiency of the sample increases. 
Information on this question might add tc the understanding of the global 
proficiency vs* discrete-point language testing controversy. There is 
some evidence to support this hypothesis, particularly when one contrasts 
the factor structure of the TOEFL for Farsi speakers with the considerably 
more complex structure for Germanic speakers* The factor-analytic 
literature on mental abilities contains a number of studies suggesting 
that mental abilities are more highly differentiated in samples that are 
older or that have had more exposure to relevant instruction and experience* 

Pike (1979) in his important monograph, "An Evaluation of Alter- 
native Item Formats for Testing English as a Foreign Language," contrasted 
intercorrelations among the five item types that constituted the then 
five-part TOEFL, for Peruvian, Chilean, Japanese, ana all TOEFL 
candidates. Table 7, after Piker's Table 7, gives the median correlations 
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Table 7 

Correlations Among TOEFL Sections as Reported by Pike 



Subtest 





Listening English 
Comp, Structure 


Written 

Expression 


Vocab. 


Read in 
Comp. 




Hispanic (n«243) 








Listening Comp. 


[.94] 


(.93) 


( -87) 


(.84) 


(.81) 


English Structure 


.85 


[91] 


( .97) 


/ OCX 

( .85) 


(.02) 


Written Expression 


.79 


.86 


[.88] 


(.88) 


(.90) 


Vocabulary 


.74 


.74 


.76 


[85] 


(.93) 


Reading Comprehension 


.71 


.71 


.71 


.78 


[.83] 




Japanese (n=199) 








Listening Comp. 


* [85] 


(.80) 


(.65) 


(.61) 


(.74) 


English Structure 


.65 


[.78] 


(.82) 


( .78) 


/ Q C\ 

( .o5) 


Written Expression 


.52 


.63 


[.76] 


(.84) 


(.84) 


Vocabulary 


.51 


.62 


.66 


[82] 


(.88) 


Reading Comprehension 


.56 


.62 


.61 


.66 


[69] 




World 


(n=l,000) 








Listening Comp, 


[88] 


(.74) 


(.67) 


(.63) 


(.77) 


English Structure 


.63 


[83] 


(.93) 


(.82) 


(.83) 


Written Expression 


.58 


.77 


[82] 


(.81) 


(.83) 


Vocabulary 


.54 


.68 


.67 


[83] 


(.84) 


Reading Comprehension 


.64 


.67 


.67 


.68 


[79] 


[ ] reliabilities 

Note. Correlations above the diagonal 


have been 


corrected 


for attenuation. 
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for the two Hispanic groups, for the Japanese group, and for the "World" 
group (a sample of all candidates)* 

The underlined corrected correlations, r , between Structure and 
Written Expression, and r^^ between Vocabulary and Reading Comprehension, 
formed the basis for a conjecture and a recommendation in the Pike 
report. The conjecture, based on corrected correlations r « .97 
and » .93 for Hispanics, vs. r. « .82 and - •88 for Japanese 
speakers, was that "the relationships among component English language 
skills may tend to be lower for Japanese students than for those having 
an Indo-European language background" (p. 55). The recommendation 
was that since Structure and Written Expression form one cluster 
(r « ,93 for World) and the Reading Comprehension and Vocabulary 
measures form another (r^ ■ .84) that "these and other considerations 
suggest a revised TOEFL Raving several components, but yielding only 
three scores: I. Listening Comprehension, 11. English Structure and 
Writing Ability, and III. Reading Comprehension and Vocabulary in 



to the conjecture, since none of the non-/ Indo-European Iflfnguage groups 
examined exhibited the clustering of Vocabulary and Reading Comprehension 
found in the Spanish and Germanic groiips. The recommendation is supported 
in the case of the English Structure and the Written Expression measures, 
but considerably less strongly supported in the case of Reading Comprehen- 
sion and Vocabulary. For non- Indo-European speakers, Reading Comprehension 
appears to be more strongly related to Structure and Written Expression 
than to Vocabulary. Indeed, the "World" corrected correlations presented 
in the Pike report for Vocabulary (.81 with Written Expression, .82 with 
Structure, and .84 with Reading Comprehension) do not offer convincing 
evidence that Vocabulary is significantly more related to Reading Compre- 
hension than to the other two measures. Although the corrected correlations 
within the two clusters exceed .90 in the Hispanic groups, this is the 
case only for the Structure and Written Expression cluster for the 
"World" group, and for neither cluster for the Japanese group in the Pike 
data. 

If analysis of TOEFL forms more recent than form YTF4 confirms 
that Reading Comprehension items indeed covary with Structure and Written 
Expression more strongly than with Vocabulary in non-Indo-European 
group3, an important consequence would emerge in interpreting TOEFL 
scores for members of such groups: In the case of discrepancies oetween 
Structure and Written Expression (Section II) total scores and Reading 
Comprehension & Vocabulary (Section III) total scores, non-Indo-European 
candidates are probably more discrepant in vocabulary than their Section 
III scores indicate, since half of this Section III score is based on 
Reading Comprehension, a shill which tends to pull the combined score 
toward the Structure and Written Expression score more than it does for 



Context. 
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Indo-European speakers. This structural difference does not affect total 
TOEFL scores, but It has considerable implications for the interpretation 
of section scores among different groups. The difference in the relation- 
ship of vocabulary to other measures in the two broad language groups may 
be partly due to the different store of cognate and false cognate words 
among Indo-European vs. non-Indo-European languages, but it is probably 
cfue more to structural and syntactic similarities and differences. 
In the absence of evidence to the contrary, we concur with Fillmore 
(1968) that human semantic, Jio re than syntactic, catego,ries share elements 
of universality. The struc^fcal differences rioted here should thus be 
taken into account in interpreting TOEFL scores, but not over interpreted. 
A relatively high Section III score for a non-European candidate probably 
represents unusually high vocabulary achievement. A relatively low 
Section III score, when compared with the Section II score, probably 
reflects low attainment in vocabulary relative to reading comprehension 
ability. However, vocabulary in a second language is acquired more 
rapidly than i« syntactic competence, and is thus more easily remediated 
for these candidates if grammatical knowledge, as measured by Section II, 
and Reading Comprehension, as measured by Section III, in are adequate. 
These considerations suggest differential interpretations of TOEFL scores 
for different language groups. They also support the position that a 
total TOEFL score may be less useful for assessing of language competence 
than the three section scores are, if Interpreted in the light of the 
language background of the students, further, if these findings are 
confirmed in analyses of other forms of TOEFL, the potential usefulness 
of a separate vocabulary score should be considered. 

Tables 8 and 9 show for samples of Foreign candidates, (a) the 
intercorrelations among item types and sections on TOEFL form YTF4 and 
(b) the average correlations for six more recently developed TOEFL forms. 
In both cases, the correlation between Sections I and II is .70, as is 
that between Sections I and III. However, the correlation between 
Sections II and III drops slightly from .83 in Form YTF4 to an average 
.78 in the Spring 1980 3CTF series. In the form analyzed in this study, 
(YTF4) the disattenuated correlation between II and III is .94, those 
between I and II and between I and III are .80 and .79, respectively. 
Thus, in Form YTF4, Section III is so strongly related to Section II that 
it impairs its usefulness as a separate score. In Spring 1980 the 
corrected correlations between I and II was .80, and between I and III, 
.77. Thus they are comparable to those of Form YTF4; however, the 
average corrected correlation between II and III has dropped slightly to 
.89. When Spring 1980 item-type subscore correlations are corrected for 
attenuation, Reading Comprehension relates almost as strongly to Structure 
(£ - .83), and to Written Expression (£ ■ .84), as to Vocabulary (£ - .86), 
although it is combined only with Vocabulary scores to create Section III 
scores. However, the corrected correlations of Vocabulary with Structure 
(.85) and with Written Expression (.83) are comparable to the corrected 
correlation of .86 with Reading Comprehension. The corrected correlation 
between Structure and Written Expression is .89, very slightly greater 
than that for other pairs of item types. Therefore, both the factor- 



■ Table 8 
Section Intercorrelations n=1135 
FORM YTF4 November 19/6 




istening 
P-. 

Structure 

Written 
Expression 



Listening 
Comp, 



• 90 



Structure 



Written 
Expression 



II 

Structure 

and 
Written 
Expression 



Reading 
Vocabulary Comp. 



Ill 
Vocabulary 
and 
Riding 
Comp, 



.64 
.71 



.66 
.69 

.82 



.70 
(.87) 

(.96) 



.61 

.67 

.70 



.70 
.69 

.77 



.71 
.73 

.79 



II * Structure and. 

Written Expression 

Vocabulary 

Read Ing 
Comp. 



,87 



r 



,74 



,83 



.80 
.72 

.88 



.83 
(.92) 

- (.93) 



III * Vocabulary and 
Reading Comp. 



.92 



Note , k^llabllltlea In main diagonal (part-whole correlations In parenthesis) 



0 



Table 9 

Average Section Intercorrelatlons, Six TOEFL Forms 
February - July 1980 



Listening Written 
Comp. Structure Expression 



II 

Structure 

and 
Written 
Expression 



Vocabulary 



111 
Vocabulary 
and 

Reading Reading 
Comp> Comp, 



II 



Listening 
Comp. 

Structure 

Written 
Expression 

Structure and 
Written Expression 

Vocabulary 

Read ing 
Comp* 



.894 



.626 
.709 



.649 
.662 

.784 



.698 
(.858) 

(.953) 
.856 



.623 
.653 

.671 

' .722 
.842 



.662 
.630 

.671: 

.714 
.710 

.806 



.687 
.692 

.725 

. .777 
(.927) 

(.921) 



I 

O 
I 



III ■ Vocabulary and 
Reading Comp. 



.899 



Note. Reliabilities In main diagonal (part-whole correlations In parentheses), 
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analytic results and the operational-section statistics seem to concur 
that the marriage of TOEFL Vocabulary and Reading Comprehension scores to 
form Section III may not have been "made in heaven.- If separate subscores 
are deemed to be important, it would appear that for non-Indo-European 
native speakers. Section III results might be more usefully reported as 
Vocabulary and Reading subscores than as a combined score. The slight 
trend over time toward decrease in correlations of Sections II and III 
apparejntly results from improved test specifications since the development 
of Form YTF4, but the trend would have to continue if the combined 
Section III scores are to become above psychometric reproach. 



Factor Extension Analysis of Background Variables 

Profiles of native language groups on background variables reveal 
similarities and differences pertinent to comparing factor structures 
across groups. Each of the seven groups in the study is predominantly 
male, although African and Arabic groups are distinctly so, with 87 
percent and 89 percent of their respective examinees being male. Groups 
seem to differ more, however, with respect to the reasons for taking 
TOEFL. For all groups, the reason most often cited is support of under- 
graduate or graduate admissions. The proportion of examinees citing 
other choices (e.g., professional licensure or employment requirement) 
varies across groups. Germanic, Farsi, Spanish, and Japanese, for 
example, are similar in containing fewer undergraduates than graduates, 
roughly in a ratio of 3 to 5 in each grcup. Chinese and African, on the 
other hand, contain more undergraduates than graduates, in approximate 
ratios of 3 to 1 and 8 to 1, respectively. In addition, three groups, 
Arabic (21 percent), Japanese (17 percent), and Chinese (17 percent) 
contain a notable proportion of examinees taking TOEFL for reasons other 
than undergraduate or graduate admissions, such as professional licensure, 
business requirements, or admission to some other school program. A 
third background variable, previous TOEFL administrations, also reveals 
some marked contrasts among language groups. Only the Germanic and 
African groups have fewer than ten percent of the examinees with previous 
TOEFL experience. In other groups, the percentages range from 14 (for 
Spanish) to 42 (for Japanese). Finally, all groups are similar in having 
a majority of examinees destined for degree programs, although Germanic 
and Japanese groups have smaller percentages in this category than do the 
others. Thesa data are summarized in Table 10. 

Those variables most clear in reflecting group differences are 
"Reasons for Taking TOEFL"~largely a measure of graduate/undergradu- 
ate status and confounded with age—and "Previous TOEFL Administra- 
tions." The Chinese and African groups differ from others on the first 
of these variables, and the Japanese, African, and Germanic groups differ 
on the second. 

Table 10 also shows the locations of factor extension correlations 
of at least + . 20 in magnitude and provides a rough idea of those 
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Table 10 

Locations of Extension Correlations Greater Than or Equal to .20 



Factor 



Variable 



II 



III 



IV 



Language Group 



Sex 



-.27 



German 
Spanish 
Farsl 
African 
Chinese 
Japanese 
Arabic 
TOTAL 



(-) 
(0) 
(0) 
(+) 
(-) 
(0) 
(+) 



Birth Year 



.22 



.21 



-.21 
-.31 
-.25 
-.24 



-.27 



Undergraduate 



.26 
.27 



-.26 
-.21 
-.23 

-.21 



-.32 



German 
Spanish 
Farsl 
African 
Chinese 
Japanese 
Arabic 
^__TOTAL^_ 
German 
Spanish (0) 
Farsl (0) 
African (+) 
Chinese (0) 
Japanese. (0) 
Arabic (0) 
TOTAL 



Graduate 



-.27 



.20 
.23 
.27 

.21 

.26 



.22 



German 
Spanish 
Far si 
African 
Chinese 
Japanese (0) 
Arabic (0) 
TOTAL 



(0) 
(0) 
(-) 
(0) 



Studying for 
Degree 



7 



-) 
(+) 
(+) 
(+) 
(+) 



Previous TOEFL 



.22 



German 
Spanish 
Farsl 
African 
Chinese 
Japanese (-) 
Arabic (0) 

TOTAL^_ 

'German ^ Z-J 
Spanish (0) 
Farsl (0) 
African (-) 
Chinese (0) 
Japanese (+) 
Arabic (0) 
TOTAL 



Key: - • Group 
0 - Group 



low on this variable 
average on this variable 
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background variables that may influence the interpretation of factor 
structures across groups. Here the most obvious trend is the occurrence 
of extension correlations of specified magnitude on variables related to 
age and/or graduate/undergraduate status. Such correlations occur 
consistently on Factor III, regardless of group profiles on those 
variables, and suggest a rather uniform relationship between the combi- 
nation of age and academic status and Factor III. In particular, age and 
graduate are positively correlated with Factor III, suggesting that 
vocabulary develops with experience/exposure. 

The remainder of Table 10 displays mcuh of variation in the way that 
background variables correlate with factors common to the seven language 
groups. For the age/academic status combination mentioned above, Spanish, 
Farsi, African, and Arabic groups have extension correlations on at least 
two of the three-'other factors. Sex appears to be another variable that 
correlates differentially across groups, ; with females, e.g., showing 
superior knowledge of vocabulary, especially in the Japanese group. 
Although these are precisely the kinds of differences that may account 
for contrasting factor structures, it is not clear that the clustering of 
groups is the same pattern of clustering demonstrated by the extension 
correlations. That is, groups that are similar in profile on a given 
background variable like graduate status do not necessarily have a 
similar pattern of extension correlations. More detailed factor extension 
correlation matrices are presented in Appendix B. 



Other Approaches 

The present study is necessarily somewhat restricted in its approach. 
The restrictions apply mainly to the use of exploratory factor analysis 
with orthogonal rotations. Alternative confirmatory factor analytic 
approaches (e.g.. Rock & Werts, 1979) could have been used to test the 
goodness of fit of TOEFL data to a factor structure specified a priori. 
Oblique rotations, in which factors are not constrained to be uncorrelated , 
would also have been appropriate, given that to require language proficiency 
factors LO be totally independent gives descriptive economy at some cost 
in substantive interpretability . Item response theor^j might also have 
been applied to look at the relationships among component skills for 
language groups. Finally, the inclusion of marker variable tests (i.e., 
tests measuring several language skills in addition to those included in 
TOEFL), such as were included in some of the previous studies mentioned 
in Oiler (1979), would have facilitated interpretations of the various 
factors. However, even within these limitations, the impressively 
consistent replications from all the included non-Indo-Eu^^opean language 
groups suggest that Vocabulary and Reading Comprehension as measured by 
TOEFL do not fono a single dimension, a finding that has implications for 
score interpretation. In particular, when a non-Indo-European speaker 
exhibits Section III scores (Reading Comprehension and Vocabulary) 
discrepant from his or her Section II scores (Structure and Written 
Expression), the difference is likely to be due more to vocabulary than 
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to reading comprehension and to be greater than the score difference 
Implies. This situation exists because part of the Section III score Is 
based on Reading Comprehension, and In these language groups, Is more 
strongly correlated with Section II, hence bringing the average Section 
III score closer to the Section II score than happens with scores earned 
by the Indo-Europeans. That non-Indo-Europ6an Section III score differen- 
tials can be identified more closely with vocabulary competency than with 
reading ability offers some useful diagnostic information for teachers of 
English as a second language* Vocabulary is relatively easily developed 
once grammar and syntax have reached an adequate stage of performance* 
Thus if a non-Indo-European exhibits relatively lower Section III scores 
on TOEFL, vocabulary development, rather than work on reading comprehension 
and vocabulary, may be Indicated* For such students reading comprehension 
appears to be measured as ruch by Section II as by Section III* This 
would suggest that Japanese students, who rank sixth on Section II but 
fifth on Section III, might be stronger in vocabulary than their section 
scores suggest and might be more in need of Structure, Written Expression 
and Reading Comprehension exercises, West Africans, in spite of their 
ranking third on Section III, are likely to be in more relative need of 
vocabulary development* Spanish speakers, on the other hand, appear from 
their means and from the factor interpretation to be relatively more in 
need of explicit structure and written expression training, but less in 
need of a vocabulary or reading comprehension focus until their syntactic 
and grammatical competence catch up with vocabulary and reading* The 
vocabulary and reading of these Spanish speakers do relate in such a 
way that a single Section III score can be a useful summary of both 
skills* 



Summary 

The present factor-analytic study was undertaken to determine the 
explanatory constructs or component abilities underlying performance on 
the TOEFL and to determine the similarity of these component abilities 
for several major language groups* The results were intended to be 
useful for the interpretation of TOEFL scores and for future test develop- 
ment activities* 

Procedures involved drawing samples of candidates from seven major 
language groups: African, Arabic, Chinese, Farsi, Germanic, Japanese, 
and Spanish* Correlations among all items on the TOEFL were computed for 
each language group and for the total sample of candidates* Separate, 
uncorrelated-factor solutions were obtained for each language group and 
for the total group of examinees* Because an infinite number of rotations 
of factors can be chosen that fit equally well the observed correlations 
among variables, a four-factor target matrix was specified, toward which 
the solution of each language^ group was rotated for best fit* Factor 
extension analysis was employed to assess the relationshipo of several 
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candidate background characteristics with factors underlying test perfor- 
mance. This technique was used to facilitate the interpretation of 
factors. 

Three or four factors appeared to be necessary for each language 
group in order to explain the bulk of the variability in test-score 
performance on the TOEFL. There was some indication that the factor 
structure of the test might be more complex for Germanic speakers, for 
whom the test is generally relatively easy. Although the nature of the 
additional components of t^est variance for this group was not readily 
interpretable, the presence of these components suggests that English 
language proficiency may be more highly differentiated for Germanic 
speakers. On the other hand, for Farsi speakers, for whom the test is 
relatively difficult, the factor strucr.ure is least clear and suggests 
fewer highly differentiated factors. 

Both similarities and differences in structures between language 
groups were noted. A listening comprehension factor could be clearly 
interpreted for each group. Other factors, however, differed from group 
to group. The African, Arabic, Chinese, and Japanese groups were generally 
similar on a factor underlying performance on Structure, Written Expression, 
and Reading Comprehension items, and on another separate factor underlying 
Vocabulary items. The Spanish and Germanic groups were also similar on 
each of two other factors, which correspond to the TOEFL subscores 
(Structure/Written Expression, and Reading Comprehension/Vocabulary) 
that are reported. 

The Vocabulary factor exhibited positive correlations with age and 
degree-intentions in nearly every language group, thus suggesting that 
vocabulary is the most likely of any of the abilities to develop with 
training or experience. In all of the language groups. Vocabulary 
items were more likely than any other item type, except Listening 
Comprehension, to form a separate ^'actor, a finding which suggests that 
including a separate vocabulary score might perhaps be justified. 

In summary, there is evidence that three major factors underlie 
performance on the TOEFL but that these factors may differ in their 
interpretation for different major language groups. The results also 
suggest rather strongly that interpretations depend on the level of 
language skills in the group being considered. 

The results also have implications for the interpretation of TOEFL 
subscores. For certain language groups, particularly the non- Indo-European 
groups, discrepancies between Section II (Structure and Written Exprejtton) 
and Section III (Reading Comprehension and Vocabulary) scores are mor^ 
likely to have resulted from knowledge (or lack of knowledge) of English 
vocabulary than from reading comprehension ability when compared with 
Germanic and Spanish speakers. Thus, a relatively low Section III score 
for non- Indo-European groups may be less critical for these groups, since 
vocabulary may be learned more readily than grammatical or syntactical 
structure. 
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Detailed interpretation of individual item loadings on the factors 
found in this study, and of their relation to specific vocabulary 
or syntactic features of each item in each language might be one way of 
controlling the structure of an English proficiency test to make that 
structure invariant across the native languages of the examinees. We 
believe that John Ciardi's recent remark about etymology also applies to 
this incredibly complex task. Ciardi said, "The maiu trouble with 
etymology is that the minimum requirement for doing it is omniscience." 



ERIC 



-27- 



Referenced 



Alderman, D. L. TOEFL Item performance across seven language groups . 
Report in progress to the TOEFL Research Committee » 1980. 

American Psychological Association. Standards for educational and 
psychological tests . Washington, D.C.: Author, 1974. 

Angelis, P. J., Swlnton, S. S., & Cowell, W. R. The performance of 
non-native speakers of English on TOEFL and verbal aptitude tests . 
(TOEFL Research Report No. 3.) Princeton, N.J.: Educational Testing 
Service, 1979. 

Angoff, W* H., & Sharon, A. A comparison of scores earned on the Test of 
English as a Foreign Language by native American college students and 
foreign applicants to U*S. colleges . (ETS Research Bulletin 70-8.) 
Princeton, N.J.: Educational Testing Service, 1970. 

Browne, M. W. , & Kristof, W. On the oblique rotation of a factor matrix 
to a specified pattern. Psychometrika , 1969, 34(3), 375-394. 

Clark, J. L. D. The performance of native speakers of English on the 
Test of English as a Foreign Language . frOEFL Research Report No. 2). 
Princeton^ N.J.: Educational Testing Service, 1977. 

Dwyer, P. S. The determination of the factor loadings of a given test 
from the known loadings of other tests. Psychometrika , 1937, 2^ 
173-178. 

Fillmore, C. J. The case for case. In E. Bach and R. T. Harms (Eds.), 
Universals in linguistic theory . New York: Holt, Rinehart and 
Winston, 1968. 

Harman, H. H. Modern Factor analysis (3rd rev. ed.). Chicago: The 
University of Chicago Press, 1976. 

Hlnofotls, F. A. B. An investigation of the concurrent validity of cloze 
testing as a measure of overall proficiency in English as a second 
language. Unpublished doctoral dissertation, Southern Illinois 
University, 1976 (Reported in Oiler, J. W. Language tests at school: 
A pragmatic approach. London: Longman Group Limited, 1979.) 

Oiler, J. W. Language tests at school: A pragmatic approach . London: 
Longman Group Limited, 1979^ 

Oiler, J. W. Evidence for a general language proficiency factor: An 
expectancy grammar. Die Neuren Sprachen , 1976, 2_, 165-174. (Reported 
in Oiler, J. W. Language tests at school: A pragmatic approach . 
London : Longman Group Limited, 1979.) 



-28- 



Oller, J. W. , & Hinofotis, F. A. B. Two mutually exclusive hypotheses 
about second language proficiency; Factor analytical studies of a 
variety of language tests . Paper read at the Winter Meeting of the 
Linguistic Society of America, Philadelphia, Pa. (Reported in Oiler, 
J, W. Language tests at school:^ A pragmatic approach . London: 
Longman Group Limited, 1976.) 

Pike, L. W. An evaluation of alternative item formats for testing 

English as a foreign language (TOEFL Research Report No. 2). Princeton, 
N.J.: Educational Testing Service, 1979. 

Powers, D. E., & Swinton, S. S. The factor structure of the GRE Aptitude 
Test: Implications for test development and assembly. (RM 76-2) 
Princeton, N.J.: Educational Testing Service, 1976. 

Rock, D. A., & Werts, C. E. Construct validity of the SAT across popula- 
tions — An empirical confirmatory study (College Board RDR 78-79, No. 5; 
and ETS RR-79-2). Princeton, N.J.<^ Educational Testing Service, 1979. 



-29- 



Appendlx A 

Four-Factor Solution Rotated to Target for 
Each Language Group 
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Table A-1 

Rotated Factor Loadings 
African Sample 





I 


II 


III 


IV 


Difficulty 


I-l 


.41* 


.26 


— 


— 


.88 


1-2 


.50* 


— t 


— 


— 


.52 


1-3 


.40* 


— 


— 


— 


.75 


I-A 


.39* 


— 


— 


— 


.74 


1-5 


.25 


.39* 


— 


— 


.76 


1-6 


.38* 


.25 


— 


— 


.52 


1-7 


.33* 


— 


— 


— 


.69 


1-8 


.50* 


— 


— 


— 


.61 


1-9 


.29* 


.29 


.24 


— 


.74 


I-IO 


.45* 


— 


— 


— 


.64 


I-ll 


.56* 


.21 


— 


— 


.66 


1-12 


.38* 


— 


.27 


— 


.45 


1-13 


.48* 


— 


— 


— 


.85 


1-14 


.40* 


.22 


— 


— 


.57 


1-15 


.48* 


— 


— 


— 


.35 


1-16 


.29* 


' -.21 


— 


— 


.33 


1-17 


- .35* 





— 


— 


.53 


1-18 


.33* 





— 




, .40 


1-19 


.40* 


.28 


.20 


— 


- .59 


1-20 


.34* 


.25 


— 


— 


.34 


1-21 


* 


— 


— 


— 


.41 


1-22 


.41* 


.36 


— 


— 


.91 


1-23 


.49* 


— 


— 


— 


.72 


1-24 


.41* 


.29 


— 


— 


.62 


1-25 


.43* 


— 


— 


— 


.72 


1-26 


.41* 


.27 


— 


— 


.75 


1-27 


.53* 


.22 


— 


— 


.68 


1-28 


.25* 








.67 


1-29 


.44* 




.26 




.33 


1-30 


.45* 


.21 






.58 


1-31 


.47* 




.25 




.33 


1-32 


.53* 








.40 


1-33 


.47* 




.24 




.75 


1-34 


.42* 








.52 


1-35 


.38* 




.24 




.57 


1-36 


. 35* 




.27 




.65 


1-37 








* 


.65 


1-38 


.25 


.33* 


.21 




.83 


1-39 


.22 




.34* 




.18 


1-40 


.24 


.27* 


.25 




.72 


1-41 


.29 


.30* 


.24 




.55 



_ * 

(Cont'd.) 

*Largeat loading of items 
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Table A-1 

Rotated Factor Loadings ^ .| 
African Sample 

4 

(ContMQ . 

I II III . . ' IV . Difficulty 



1-42 


.29* 


1 ^ 






.63 


1-43 


.52* 


.23 






.44 


1-44 


.41* 


.^8 






.55 


1-45 


.28* 


.22 


:27 


-.26 


.83 


1-46 


.31* 








.32 


1-47 


.34* 








.35 


1-48 


.35* 






.22 


.31 


1-49 


.49* 


.27 






.64 


1-50 


.32* 








.75 




*Largest loading of Items 
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Table A-1 



Rotated Factor Loadings 
African Sample 



I II III IV ' Difficulty 



II-l . 31 .33* .22 — -.92 

II- ^ — .29 .39* — .87 

1 1-3 .25 .42* — -.42 .94 

11^4 — .34* -- — .96 

1 1-5 ~ .34* .20 ~ .83 

II-6 ~ ~ .35* ~ .76 

II-7 ~ .45* -- — .95 

1 1-8 ~ .21* — ~ .69 

1 1-9 — .32 ' .57* .68 

II-IQ — .45* .23 — .87 

II-ll .27 . .28 .34* — .67 

■ 11-12 . ~ .49* — — .85 

11-14 .33 .20 .35* — .40 

11-15 — .33* — — .77 

II-IS ~ .64* — -.44 .97 

.11-17 ~ .66* — . -.24 .93 

11-18 — .29* ' — — .80 

11-19 • ~ .46* .28 — .72 

11-20 .22 .27* ~ ~ .65 

11-21 ■ — ,38* ~ ~ .83 

11-22 .20 .74* — -.24 .92 

11-23 .25 ■ .47* — — .73 

11-24 ~ .43* ~ — .68 

1^25 ~ .53* .34 ~ .61 

11-26 .23 .42* .42 --• - .51 

11-27 . • ~ .58* — ~ .84 

11-28 — .62* — ~ .88 

11-29 ~ .82* ~ — .91 

. 11-30 -- .65* ~ — .82 

11-31 — ' . .32* ~ — .52 

11-32 '. -_ ,22* ~ ~ .45 

11-33 , ~ -.67* ~ — .82 

11-34' .22 .31* .30 ~ .50 

11-35 '. .24 . .45* .23 — .61 

11-36 ' .57* .08 — .72 

11-37 , — , .41* — .45 

11-38 ' ~ .29* .25 -- .47 

11-39 ~ .3^* — — .56 

• 11-40 .27 . . .35* .23 ~ .58 



*Largeat loading of items 
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Table A-1 

Rotated Factor Loadings 
African Sample 



1 11 



Ill-l — .23 

III-2 ~ .45* 
I II- 3 

■ III-4 ~ .29* 
III-5 

III-6 — .28 

I1I-7 .20* 
1 1 1-8 

III-9 .42 .27 
III-IO ^ 

III-ll ~ . 26* 

III-12 ~ * 

III-13 — .33* 

III-14 — . 33 

• III-15 — .26 
III-16 
III-17 

III-18 .33 .36 

III-19 .28 .30* 

III-20 ~ . 21 

III-21 — .24 

III-22 ~ . 24* 

III-23 — .21 

III-24 ~ 

III-25 — .40 

III-26 ~ .22 

III-27 — .37 

LII-28 .11 

III-29 .20 .22 

III-30 .27 .21 

III-31 — • .43* 

III-32 -- .23* 

III-33 .29* 

III-34 .23 .23* 

III-35 .28 .32* 

III-36 .28 .29* 

111-37 — .35* 

III-38 .20 .27 

III-39 .28 .34* 
III-40 

III-41 .21 .37* 



♦Largest loading of items 



III IV Difficulty 

.49* — .71 

. 97 

.24* — .80 

.28 — • 66 

.41* — .68 

.43* — .67 

. 70 

.44* ~ .77 

.43* — '77 

-.20* .91 

.26 -- . 77 

.69 
.88 

.34* — .90 

.39* — .56 

.29* — .77 

* .43 

.37* - .77 

.25 - ■ -81 

.27* — .53 

. 35* — • 66 

.21 " - -79 

.39* — .81 

.62* — .51 

.61* — .52 

.45* — .75 

.46* — .64 

.45* -.27 .47 

.31* — .43 

.67* — .24 

.28 — .80 

.23 ~ .95 

.20 -.24 .45 

.80 
.85 
.63 

.30 — .83 

.33* — .74 

.23 ~ .69 

.21* — .77 

.34 ~ .74 



(Cont'd.) 
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Table A-1 

Rotated Factor Loadings 
African Sample 

(Cont'd,) 

1^ III IV Difficulty 



III-A2 


— 


— 


.26* 


— 


.74 


III-A3 


— 


.35* 


— 


— 


.88 


III-AA 


.30 


.43* 


.22 


-.22 


.71 


III-A5 


.25 


.26 


.35* 




.63 


T T T / C 


. Zo 


. 48* 






.90 


III-47 


.24 


.54* 






.80 


III-48 


.21 


■ 24* 


.23 




.45 


III-49 


~_2 


.42* 




. z5 


.81 


III-50 






* 




.55 


III-51 


.30 


. 15* 


— 


.27 


.46 


III-52 


.23 


. 39* 




-.20 


.50 


III-53 








.30* 


.55 


III-54 


.29 


.45* 




.31 


.74 


III-55 


.27 


.49* 




.32 


.72 


I I 1-56 


.26 






.42 


.65 


III-57 


.25 


.23 


c23 


.29* 


.38 


III-58 


.28 


.44* 




.32 


.59 


III-59 


.30 




.34* 


.21 


.41 


III-60 


.3: 


.35* 




.34 


64 


Variance 


11.67 


15.82 


9.07 


2.96 




Coefficients 












of Congruence 


.981 


.957 


906 







*Largest loading 'of items 
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Table A-2 



Rotated Factor Loadings 
Arabic Sample 





I 


II 


III 


IV 


Difficulty 


I-l 


.36 


.39* 


— 


— 


.90 


1-2 


.41* 


.24 


— 


— 


.61 


;-3 


.45* 


.26 


— 


— 


.70 


7.-4 


.4S* 


.22 


— 


— 


.81 


1-5 


.21 


.44* 


— 


— 


.80 


.1-6 


.39 


.39* 


.20 


— 


.62 


1-7 


.42* 


.28 


— 


— 


.69 


1-8 


.41* 


.30 


— 




.69 


1^9 


.23 


.28* 





— 


.69 


I-IO 


.44* ■ 


.36 


. 25 


— 


.53 


I-ll 


.46* 


.39 


.28 


— 


.59 


1-12 


.53* 


.25 


.21 


— 


.62 


1-13 


.48* 


.47 





— 


.87 


1-14 


.48* 


.37 





— 


.64 


1-15 


.39 


.41* 





— 


.55 


1-16 


.44* 





.20 


.23 


.26 


1-17 


.50* 


.27 


.24 




.57 


1-18 


4- 


.20* 




— 


.42 


1-19 


I 


.27* 


.20 


— 


.45 


1-20 


.26 


. 34* 


.2 


— 


.35 


1-21 


.43* 






— 


.76 


T-22 


.46* 


.46 


— . 


-20 


.92 


1-23 


.44* 


.38 







.88 


1-24 


.37* 


.34 


— * 


— 


.75 


1-25 


.50* 


.22 


— 


— 


.77 


1-26 


.49* 


.36 





— 


.82 


1-27 


.56* 


.36 


— 


— 


.79' 


1-28 


.41* 








.76 


1-29 


.41* 


'.35 






.64 


1-30 


.60* 


.32 






.73 


1-31 


.68* 


.23 


.24 




.51 


1-32 


.51* 


.28 






.49 


1-33 


.45* 


.32 






.70 


1-34 


.52* 


.30 






.64 


1-35 


.36* 


.33 






.58 


1-36 


.43* 








.74 


1-37 




.22* 






.51 


1-38 


.36 


.37* 


.22 




.77 


1-39 


.32* 




.32 




.22 


1-40 


.43* 


.32 


.23 




.74 


1-41 


.23 


.34* 


.28 


-.21 


.57 



^Largest loading of Items 
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Table A-2 

Rotated Factor Loadings 
Arabic Sample 





I 


II 


III 


IV 


Difficulty 


1-42 


.42* 


.28 






.70 


I-A3 


.43* 


.36 






.53 


I-AA 


.44* 


.25 


.21 




.75 


I-A5 


.50* 


.27 


.21 


-.20 


.82 


I-A6 


.22 


.24* 






.38 


I- A 7 


.39* 


.20 


.32 




.50 


I-A8 


.50* 


.29 


.26 




.45 


I-A9 


.57* ' 


.31 


.28 . 




.68 


1-50 


.46* 


- .38 


.21 




.64 



/ 



- ' ^ 

*Largfts,t loading of items 
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Tabla A-2 

Rotaced Factor Loadings 
Arabic Sample 





I 


11 


III 


^IV 


Difficulty 


II-l 


.38* 


.31 


.20 


— 


■^'.80 


II-2 


.31* 


.21 





— 


.72 


II-3 


.30* 


.21 


.29 


-.26 


.93 


II-4 


.22 


.37* 


.25 


— 


.81 




.33 


.36* 


— 


— 


.77 


1 1-6' 




.22* 


.20 


— 


.67 


II-7 




. .29* 


— 


— 


-»5'7 


1 1-8 


.33* 




— r 


-.30 


.46 






.25* 


. — 


— 


• .41 


11-10 


.39 


.44* 


.36 


— 


.53 


11-11 


.26* 


.26 


.23 


— 


.54 


11-12 






.36* 


— 


.36 


11-14 




.44* 


.29 


— 


.17 


11-15 


.27 


.44* 


.39 




.45 


11-16 


.40 


.64* 






.94 


11-17 


.36 


.64* 




-.31 


.90 


11-18 




.39* 


— mm 





.65 


11-19 





.34* 


.26 





.58 


11-20 





.37* 






.78 


TT-21 


.32 


.50* 




— mm 


.76 


11-22 


. 32 


.52* 


.26 


-.20 


.80 


XX 




.52* 







.73 


11-24 




.42* 


.27 


— mm 


.56 


11-25 




.45* 


.25 




.64 


11-26 




.50* 


.36 




.50 


11-27 




.54* 




. -.20 


.76 


11-28' 




.48* 






.76 


TT — 90 
1 i— Z!f 


99 


52* 


.30 


-.30 


.75 


11-30 


.22 


.32* 


.31 




.52 


11-31 


.24 


.36* 






.50 


11-32 




.44* 




-.25 


.32 


11-33 




.64* 






.66 


11-34 




.34* 






.39 


11-35 




.37* 


.36 




.49 


11-36 


.25 


.40* 


.20 




.38 


11-37 




.57* 


.25 




.42 


11-38 




.32* 


.30 


.21 


.29 


11-39 




* 






.26 


11-40 




.42* 


.21 




.26 


*Largesc 


loading of Items 




y 


* 





ERIC 



-38- 



Table A-2 

Rotated Factor Loadings 
Arabic Sample 





I 


II 


III 


IV 


Difficulty 


III-l 


— 


.28* 


.23 


— 


.83 


J.II-2 


. 25 


— 


.42* 


— 


.85 


I J. I- 3 


.33 


.27 


.45* 


-.27 


.70 


III-4 


.31 


.35 


.48* 


— 


.66 


II 1-5 


— 


— 


.32* 


— 


.53 


III-6 


.26 


.38* 


— 


-.27 


.92 


III-7 


.27 


-- 


.35* 


— 


.81 


1 1 1-8 


— 


.26 


.38* 


/' 


.63 


1 1 1-9 


.48* 


.32 


.32 


/ 


.71 


III-IO 





— 


.22 


.26* 


.67 


III-ll 





* 


— 


— 


.69 


III-12 


.39* 





— 


— 


.70 


III-13 





— 


.43* 


— 


.56 


II 1-14 





.26 


.45* 


-.24 


.50 


III-15 









.3 2* 


.36 


III-16 




.22 


.46* 


— 


.28 


III-17 






.23* 


— 


.46 


II 1-18 


.25 


.39* 


.32 . 


— 


.75 


III-19 


.24 





.32* 


.20 


.47 


III-20 







.28* 


— 


.49 


III-21 


.23 





.39* 


— 


. 32 


III-22 




.26 


.35* 


.33 


.41 


III-23 







.35* 


— 


.33 


III-24 





.37* 


.32 


— 


.35 


III-25 





.45* 


.28 


— 


.37 


III-26 





.43* 


.31 


.22 


.30 


III-27 


.21 


— 


.26* 


— 


. 30 


IIT-28 






.44* 


.38 


.22 


III-29 




.45* 


.37 




.24 


III-30 " 




* 






1.13 


III-31 


.27 


.42 


.43* 




.66 


III-32 


.34 


.37* 






.89 


III-33 






.32* 




.42 


III-34 


.27 


.28* 


.26 


-.24 


.80 


III-35 


.35* 


.29 


.24 


-.34 


.85 


III-36 


.22 


.28* 




-.20 


.65 


III-37 


.28 


.33 


.35* 


-.21 


.79 


III-38 


.30. 


.37* 


.35 


-.25 


.64 


III-39 




.41* 


.26 




.74 


III-4Q 




.29* 


.23 




.70 


III-41 


.21 


.47* 


.29 


-.29 


.73 



(Cont'd.) 

*LArg«8t loading of items 



4 »^ 

4 « 



Rotated Factor Loadings 
Arabic Sample 

(Cont'd.) ' 





1 


11 


III 


IV 


Difficulty 


III-42 


.29 


.32* 


— — 




.77 


111-43 


.20 


.36* - 





-.20 


.76 


111-44 


.20 


.51* 


.24 




.58 


111-45 


.21 


.31* 


.28 


— — 


.51 


111-46 


.34 


.39 


.41 


.41* 


.76 


111-47 




.49* 


.22 




.65 


111-48 w 




.20* 






.41 


111-49 




.39* 


.34 


-.20 


.73 


111^50 


.20 


.27 


-* 


-.23 


.64 


111-51 




.42* 


.45, 




.50. 


111-52 




.29 


.60* 




.43 


111-53 




.31 


.31* 




.63 


111-54 




.43 


.44* 




.68 


111-55 


.20 


.45* 


.33 / 




.69 


111-56 




.46* 


.40 




.61 


111-57 




.27 


.32* 


.28 


.38 


111-58 




.50* 


.40 




.41 


111-59 




.47* 


.40 




.52 


111-60 




.49* 


.20 




.60 


Variance 


13.87 


18.17 


10.37 


3.63 





Coefficients 

of Correlation -983 .947 .964 



^Largest loading of items 
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Table A-3 

Rotated Factor Loadings 
Chinese Sample 





1 


II 


III 


IV 


Difficulty 


I-l 


.56* 


— 


— 


— 


.92 


1-2 


.45* 


— 


— 


— 


.72 


1-3 


.53* 


— 


— 


— 


.70 


1-4 


.42* 


— 


— 


— 


.71 


1-5 


.25* 


.22 


.25 


.25 


.88 


1-6 


.32* 


.30 


— 


— 


.66 


1-7 


.31* 


.29 


— 


— 


.70 


1-8 


.52* 


. 20 , , , 


— 


— 


.69 


1-9 




.2f*" 


— 


— 


.79 


I-IO 


.41* 


.26 


— 


mm w* 


.48 


I-ll 


.53* 


.37 


— 





.63 


1-12 


.39* 


.28- 


— 





.57 


1-13 


.46* 


.39 


— 





.84 


1-14 


.29* 


.25 


.26 





.52 


1-15 


.38* 


.21 


.26 





.50 


1-16 


.23* 





— 


-.24* 


.20 


1-17 


.41* 


.24 


— 


— 


.61 


1-18 


.20* 


— 


— 


-.24* 


.38 


1-19 


.22* 


— 


— 





.51 


1-20 


.22 


.29* 


— 





.44 


1-21 


.38* 





— 





.59 


1-22 


.44* 


.24 


— 


.20 


.92 


1-23 


.56* 


.20 


— 


— 


.86 


1-24 


.44* 


.25 


— 


— 


.53 


1-25 


.49* 





.21 


— 


.75 


1-26 


.40* 


.37 


— 


— 


.84 


1-27 


.48* 


.29 


— 


— 


.75 


1-28 


.48* 


.27 


— 


— 


.69 


T-2Q 


. 45* 








.72 


1-30 


.49* 


.34 


— 


— 


.66 


1-31 


.49* 


.33 






.48 


1-32 


.61* 


.24 


.24 




.34 


1-33 


.36* 


.23 




-.23 


.51 


1-34 


.47* 


.28 






.65 


1-35 


.30* 


.29 






.45 


1-36 


.39* 


.25 






- .61 


1-37 








* 


.60 


1-38 


.33 


.48* 






.84 


1-39 


.22* 






-.20 


.27 


1-40 


.34 


.50* 






.84 


1-41 


.27 


.43* 






.65 


*L«rge«t 


loading of Items 








(Cont'd.) 



i 



-41- 

Table A-3 

Rotated Factor Loadings 
Chinese Sample 



(Cont'd.) 





I 


II 


III 


IV 


Dlfflcu 


1-42 


.23 




.24* 




.62 


1-43 


.43* 


.32 






.49 


1-44 


.57* 


.38 






.62 


1-45 


.34 


.37* 






.82 


1-46 


.35* 


.25 






.36 


1-47 


.33* 


.29 


.20 




.58 


1-48 


.57* 


.23 






.42 


1-49 


.48* 


.28 






.66 


1-50 


.45* 


.25 






.70 



ilty 



^La^g^at loading of items 



V 



-42- 



Table A-3 

Rotated Factor Loadings 
Chinese Sample 





T 


11 


III 


IV 


Difficulty 




.22 


.25 


.31* 




.73 


II-2 


.20 


.23* 






.70 


II-3 


.41 


.44* 






.90 


1 1-4 


.29 


. 30* 


.28 




.73 


XX J 


.26 


.46* 






.84 


1 1-6 

XX w 






.25* 




.64 


TT-7 

XX / 


.24 


. 24* 


.20 




.65 


II-8 


.33* 


.26 






.64 


TT-Q ' 

XX 7 






. 36* 




.52 


11-10 


.40 


.46* 


.42 




.37 


TT-11 

XX XX 


.25 


.42* 


.22 


_ 


.55 


11-12 


.24 


.43* 






.30 


X X X*T 




. 37* 


.31 




.41 


XX x^ 




. 36* 


.36 





.39 


TT-16 

X X Xw 




. 54* 




.26 


.93 


TT-17 

XX X / 


.40 


.61* 




.36 


.91 


TI-18 

XX XV 


. 21 


.47* 


.20 




.82 


TT-*-l Q 




.40* 


. 32 




.58 


X X 






.21* 





.76 


X X ^x 




.44* 


.24 


.35 


.80 


TT-22 

XX 


.22 


. 63* 




.48 


.88 


xX""^ J 






.29* 




.67 


xX— 




38* 


.21 




.72 


TT— 9S 
XX— 




43* 


.20 


.29 


.81 


11-26 

X X ^ W 





. 33* 


.25 




.51 


11-27 


.29 


.53* 




.23 


.86 


11-28 


.23 




.25* 




.76 


II'29 


.31* 


.39* 


.27 




.78 


11-30 




.38* 


.32 




.55 


11-31 




.20 


.24* 




.58 . 


11-32 






* 




.29 


11-33 




.47* 


.20 




.82 


11-34 




.27 


.29* 




.63 


11-35 




.38* 


.28 




.73 


11-36 


.26 


.31* 






.49 


11-37 


.23 


.24 


. 32* 




.34 


11-38 




.22 


.31* 




.45 


11-39 


* 






-.29 


.27 


11-40 


.21 


.29* 


.25 


-.20 


.30 



*Larg«st loading of items ^ ^ 



-A3- 



t 



Table 

Rotated Factor Loadings 
Chinese Sanple 





I 


II 


III 


IV 


Difficulty 


III-l 


— 


.22 


.41* 


— 


.66 


III-2 


— 


.46* 


.22 


— 


.89 


1 1 1-3 


.22 


.42* 


— 


— 


.76 


III-4 


.21 


.29 


.36* 


— 


.61 


1 1 1-5 


— 


.25 


.29* 


— 


.55 


1 1 1-6 




.36* 




— 


.72 


III-7 




— 


* 


— 


.67 


III-8 


— 


.21 


.31* 


— 


.65 


1 1 1-9 





.28* 


.26 


— 


.70 


III-IO 







.38* 


— 


.48 


III-ll 







.27* 


— 


.65 


II 1-12 


.27 


.46* 


— 


.23 


.79 


III-13 




.32 


.34* 





.67 


III-14 




.44 


.46* 





.36 


III-15 




.2? 


.35* 


-.24 


.20 


III-16 




.30* 


.29. 




.53 


III-17 







* 


— 


.49 


III-18 


.31 


.34* 


.31 


— 


.55 


III-19 







.43* 


— 


.36 


I I 1-20 




.37* 





— 


.51 


III-21 


.25 


.26 


.31* 


— 


.57 


I I 1-22 


.27 


■ .38 


.46* 


— 


.37 


III-23 





— 


.30* 


— 


.41 


I 11-24 


— 


.25 


.27* 


— 


.41 


I I 1-25 


— 


.24 


.39* 


— 


.39 


III-26 


— 


.30 


.39* 


— 


.58 


III-27 


— 


— 


.23* 


— 


.46 


III-28 






.58* 


-.27 


.21 


III-29 


. 21 




.30* 


-.27 


.27 


III-30 






.38* 


-.24 


.13 


III-31 


.26 


.46* 


.25 




.73 


III-32 


.22 


.47* 






.89 


III-33 


.20 


.44* 






.62 


I I 1-34 


.26 


.29 


.36* 


.34 


.83 


I I 1-35 


.50 




.50 


.55* 


.92 


III-36 




.35* 


.25 


.24 


.78 


III-37 


.23 


.45* 


.25 




.82 


III-38 


.24 


.33* 






.68 


III-39 




.40* 






.75 


III-40 




4c 






.74 


III-41 


.26 


.50* 


.21 




.69 



(Cont'd.) 

' *Largest loading of iteas 



ERIC 
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Table X-3 
Rotated Factor Loadings 







Chinese Sample 




• 


(Cont d. ) 






fel ^'^ 




















I 















i 








I 11-42 




. 32* " 




. 21 


70 


III-43 


. Li. 


. 36* 






46 


III-4A 


.20 


c ^ ^ 

. 56* 


•<«Z2 




66 






. 39* 




58 


III-46 


.27 


.64* 




V 


. 00 


III-47 


. 29 


. 46* 


.27 




67 


III-48 




.25* 


. 21 




» . / 


III-49 


. 24 


.47* 






. 0*» 


III-50 




.35* 






.75 


III-51 


.21 


.57* 






.65 ■ 


III-52 


.21 


.65* 


.22 




.50 


III-53 


— 


.47* 






.72 


III-54 




.51* 






.85 .' 


III-55 


.23 


.61* 






.81 


III-56 


.23 


.55* 






.68 


I 11-57 




.23 


• . 37* 


• .3i' . 


.40 


III-58 


.21 


.64* 


' , .26 
t34 




.56 


III-59 


.21 


.55* 




.59 


III-60 


.25 


.45* 


.24 ■ 




.73 

t 


Variance 


12.76 


17.63 


8.45 


3.33 





Coefficients 

of Congruence .984 .954 .969 



*Larg€st loading of items 



-45- 



Table A-4 

.ed Factor Loadings ' ^ 

Farsl Sample 



I II m IV Difficulty 



T 1 


• 33** 


'^1 
• ji 






.84 


T 'O 

1— Z 


17* 


• /z 






. 52 


1-- J 


/■ n* 








S8 


i— ^ 




9 A 

. z^ 






. 75 












80 


T C 


An* 


1 9 






S8 


1-7 


lo * 
• JZ^ 








61 
• \j J 


T O 

I-O 


A Q * 


. Z3 


















fin 


I-IO 


• 27 


1 Q * 






* HA 


1^11 


. 51* 


- '^9 






A 7 


1-12 




. 37 






. •+ J 


1-13 


. 37* 


. 27 






• Oo 


T 1 / 

^-14 


0 <* 


9 A 






SO 


I-l^ 


A 7 * 










T 1 C 

I-lo 




7A* 




90 
• Z U 




T IT 
I-l / 


fen 


17* 




99 
* z z 


. 38 


i-io 




9A* 
. zu ** 






.41 










27* 


.41 


T on 


• Z i 


11* 






. 32 


i— Z i 


"* 7 








.73 


T O O 

i~Z z 


^ >* 


9 1 
. Z J 






.89 




• 3D** 


9 A 






.74 


T 9 A 


A (i* 


19 
. 3Z 






. 59 


i-ZD 


^ J 


1 7 






.73 


1-26 


.4/* 


.30 






.68 


1-27 


.68* 


.25 






.73 


1-28 


.50* 


. 33 






.57 


1-29 


.54* 


.43 






.55 


1-30 


.51* 


.44 






.55 


1-31 


.53* 


.46 






.35 


1-32 


.60* 


. 37 






.37 


1-33 


.44* 


. 32 






.44 


1-34 


.39* 


.29 






.55 


1-35 


.32 


.37* 






.35 


1-36 


.48* 




.23 




.58 


1-37 




* 






.51 


1-38 


.46^- 




.31 




.67 


1-39 


.28- 


.24 


.21 


.21 


.19 


1-40 


.44* 




.25 




.69 


1-41 


.40* 




.37 




.50 



(Cont'd.) 

♦Largest loading of items 

ERLC - - 



(Cont'd.) 



-46- 



- Table. A-4 

Rotated Factor Loadings 
Farsi Sample 





■ I 


II 


III 


IV 


Difficulty 


1-42 


.35* 








.67 


1-43 


.50* 


.27 






.44 


I-4A 


.51* 


.25 






.59 • 


1-45 


.37* 


.24 


.32 




.70 


1-46 


.21 


.26* 






.34 


1-47 


" .31 


.35* 




.31 


.35 


1-48 


.27 


.45* 






.40 


1-49 


.43* 


. .33 






.55 


1-50 


.55* , 


.32 


IT" 




.53 



^Largest loading of Items 



-47- 



Table A-4 

Rotated Factor Loadings 
Farsi Sample 



I II III ly Difficulty 



II-l 


.35* 


.32 


— 


— 


.74 


II-2 




.37* 


— 


— 


.63 


II-3 


.34 


— 


.37* 


— 


.83 


1 1-4 


.32* 


— 


, .26 


— 


.68 


1 1-5 


.42* 


.28 


— 


— 


.69 


II-6 


.22* 


— 


— 


— 


.65 


II-7 


.25 


.28* 


— 


— 


.44 


1 1-8 


— 


.49* 


.27 


.36 


.12 




— 


— 


* 


— 


.44 


U-10 


.40 


:5i* 


.22 


— 


.33 


II-ll 


.30 


— 


.30* 


— 


.48 


11-12 


— 


— 


.24* 


— 


.26 


11-14 


— 


.29* 


.23 


.28 


.12 


11-15 


- - . 


.32 


.42* 


— 


.30 


11-16 


.46*, 


.24 , - 


.24 


— 


.83 


11-17 


.43* 


.31 


.22 


.32 


.75 


11-18 


.25* 







— 


.67 


11-19 




.23 


.24* 


— 


.54 


11-20 


.20 





.22* 


— 


. .71 


11-21 


.39* 


.22 


.24 


.30 


.72 


11-22 


.35* 


.31 


.22 


.27 


.68 


11-23 


.24 


'■ .23 


.30* 


— 


.59 


11-24 


.25* 


.25 ' 


.23 


— 


-.44 


11-25 




.-33 


.34* 




.57 


11-26 




.35 


.42* 




.36 


11-27 


.38 


.26 


.38* 




.57 


11-28 






.31* 




.61 


11-29 


.33 


.28 


.39* 




.59 


11-30 




.31* 


.24> 




.39 


11-31 






.20* 




.39 


11-32 




* 






.21 


11-33 


.27 


.30* 


.25 




.61 


11-34 








* 


.41 


11-35 




.35* . 


.20 




.58 


11-36 




.31* 


.22 


.22 


.23 


11-37 


.26 


.39* 


■ .30 




.38 


11-38 




.22 


.25* 




.33 


11-39 




.23* 






.23 


11-40 




.31* 




.23 


.20 



*Largesc loading of Items 



ERIC 



-48- 



Vable A-4 

^ Rotated Factor Loadings 
Farsl Sample ^ 





I 


II 


III 


IV 


Difficulty 


TTT-1 


.26* 




.25 


.21 


.82 


T T T — 9 
X X X 


24 




.36* 




.74 


X X X J 


.24 




.34* 




.58 


TTT-^ 

XXX H 




. 29 


.37* 




.47 


X X X J 





. 24 


.35* 




.40 


X X X o 


.26* 








.85 


T TT — 7 
X X X / 






. 37* 




.74 


X X X o 


. 20 




.38* 




.56 




• jy 


34 


.26 




.67 




. Z X 




.46* 




.60 


T T T 11 






* 




.47 


T T T 10 






. 20 




.64 


T T T 11 
111-1 J 




9A A 
.ZD 






.38 


T T T 1 /. 










. 32 


T T T 1 C 

III-1j 


it 








.42 


T T T 1 ^ 

111-io 




9 S 


m* 




^ • 25 


T T T 17 
111—1 / 




It 






.44 


T T T "1 O 


o /. 


An* 






43 


T T T 1 O 
lll-"iy 




'^9 * 
. JZ 






. 38 


III-20 




. Z J " 






. 35 


T T T O 1 

III-21 




A 'I * 




?3 


.29 


III-22 




'I 'I *- 
. J J 


91 

. Z X 




. 37 


III-23 






* 




. 32 


T T T O /. 

Ill— 




. zu 


'^'^ * 




'.21 


T T T 0 Q 

111— Z J 




. jj 


'^A * 




.23 


T T T 0^ 

111-Zo 




^^9 * 
. JZ 


9ft 




. 26 


T T T— .0 7 
111— Z / 




90* 






.24 


III-28 




.37* 







.23 


T T T O O 

111-29 


. Z z 








. 18 


III-30 








* 


.13 


III-31 


.43* 


.29 


.31 




.57 


III-32 


.48* 




.31 




.79 


III-33 








* 


.33 


III-34 


.40* 




.28 




.78 


III-35 


* .52* 




.33 




.83 


III-36 


.38* 




.22 




.61 


III-37 


.41 




.44* 




.64 


III-38 


.25 




.42* 




.47 


III-39 




.21 


.30* 


.22 


.69 


III-4C 






.26* 




.56 


fII-41 


.43* 




.42 




.62 



(Cont'd.) 

*Largest loading of items w * 
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Table A-A 

Rotated Factor Loadings 
Farsi Sample 



(Cont'd.) 





I 


II 


III 


IV 


Dlfflcu 


I 11-42 


39 








• DO 


III-A3 


.25* 




• 22 




. 00 


III-A4 


.39* 


• 27 


• i/ 




. 52 


III-A5 


. 21 


• 2«f 


. 29** 






III-A6 


. 47 




. 53* 


" " 


.62 


III-47 


. 33 


.21 


.46* 




. 51 


III-A8 






.23* 


-T 


.47 


III-A9 




.23 


.27* 




.50 


III-50 


.25 




.28* 





.36 


TT T-Sl 


. 21 




.43* 




.36 


III-52 


.27 


.25 


.43* 





.31 


III-53 


— 


— 


.32* 





.48 


III-54 


.38 




.39* 




.57 


III-55 


.36 




.44* 




.64 


III-56 




.30 


.30* 




.55 


III-57 




.31* 


.24 




.37 


III-58 


.21 


.34 


.42* 




.32 


III-59 




.24 


.55* 




.37 


I I 1-60 


.36 




.40* 




.55 


Variance 


15.52 


10.47 


9.65 


2.78 




Coef f ic ients 












of Congruence 


.990 


.899 


.908 







^Largest loading of items 



-50- 



Table A- 5 

Rotated Factor Loadings 
Germanic Sample 





I 


II 


III 


IV 


Difficulty 


I-l 


.34* 


.27 


— 


-.21 


.97 


1-2 


.30 


.31* 


— 


-.22 


.77 


1-3 


.40* 




.32 


— 


.83 


1-4 


.37* 





— 


— 


.95 


1-5 


.20 


.24* 


.23 


— 


.96 


1-6 


.23 


.30* 


— 


— 


.81 


1-7 


.34 


.44* 


— 


— 


.90 


1-8 


.42* 


.31 


— 


— 


.94 


1-9 


.32* 


.27 


— 


— 


.89 


I- 10 


.37* 


— 


— 


— 


.88 , 


I-ll 


.41* 


.27 


— 


— 


.93 


1-12 


.48* 


.37 


— 


.24 


.90 


1-13 


.52* 


— 


— 


.30 


.99 


I-IA 


.48* 


.29 


— 


— 


.83 


1-15 


.50* 


— 


.25 


— 


.80 


1-16 


.39* 


.20 


— 


-.20 


.62 


I- -.7 


.48* 


.31 


— 


— 


.80 


1-18 


.36* 


— 


— 


— 


.60 


1-19 


.31* 


.24 


— 


— 


.83 


1-20 


.31* 


.24 




— 


.72 


1-21 


. 33* 


.25 


— 





.94 


1-22 


.39* 




— 


— 


.98 


1-23 


.47* 


. 36 




-.20 


.96 


1-2', 


.55* 







— 


.86 


1-25 


.63* 


.23 




— 


.95 


1-26 


.32* 


.24 





- - 


.96 


1-27 


.39* 


.22 




-.22 


.95 


1-28 


.46* 


.23 


— 




.91 


1-29 


.23* 








.85 


1-30 


.37* 


.27 






.95 


1-31 


.65* 


.25 






.84 


1-32 


.42* 


.28 




.21 


.69 


1-33 


.52* 


.22 




.33 


.83 


1-^4 


.48* 


.26 






.89 


1-35 


.42* 


.23 


.24 




.83 


1-36 


.36 


.41* 


.32 




.94 


1-37 


.21* 


.21 






.75 


1-38 


.24 




.30* 




i96 


1-39 


.47* 


.21 


.32 




.59 


1-40 




.35* 






.97 


1-41 


.48* 




.23 




.85 



(Cont'd.) 



*Largest loading of item 



ERIC 




Table A-5 

Rotated'^actor Loadings 
Germanic Sample 



(Cont'd.) 





.1 


II 


III 


IV 


Difficulty 


1-42 


,A4*- - 








.95 


I- A 3 


.46* 








.91 


1-44 


.22* 








.93 


1-45 


.38* 


.33 


.31 




.87 


1-46 


A 








.78 


1-47 


.51* 


.31 


.25 




.86 


1-48 


.66* 


.32 






.83 


1-49 


.64* 


.43 






.93 


1-50 


.54* 


.25 






.95 



♦Largest loading of item 



ERIC . . 60 
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Table A-5 

Rotated Factor Loadings 
Germanic Sample 





I 


II 


III 


IV 


Difficulty 


II-l 


. 37 


. 39* 







.92 


II-2 




* 







.84 


II-3 


. 26 


.43* 





.36 


.98 






.38* 


.23 


.26 


.90 


II-5 


. 20 


.35 




.41* 


.98 


II-6 


. 22 


.44* 


. 24 




.78 


II- 7 


. 25 


.38* 




— 


.86 


II-8 


.48* 


. 36 


.23 


-.21 


.59 


II-9 




.20 


. 34* 




.56 


11-10 


.27 


.49* 


.29 


— 


.86 


11-11 





.23* 


— 


— ■ 


.82 


11-12 




. 33 


.39* 


— 


.73 


11-14 





— _ 


.55* 


-.29 


.31 


11-15 





. 31 


. 34* 


-.35 * 


.45 


11-16 


.22 


.48* 


. 37 


— 


.98 


11-17 


.49* 


.40 


— 


.38 


.98 


11-18 





.46* 




— 


.92 


11-19 





.35* 


. 21 


,27 


.90 


11-20 





.45* 


— 





.91 


11-21 


.27 


.46* 


.24 





.91 


11-22 





.80* 


— 





.91 


11-23 





.42* 


.23 


.24 


.88 


II-2'' 




.25 


.31* 


— 


.64 


11-25 


-.23 


. 36* • 


. 34 





.81 


11-26 




.46* 


.45 




.81 


11-27 


.23 


.62* 


.22 




.94 


11-28 




.66* 






.92 


TT-29 


.20 


.69* 


.27 


.21 


.93 


11-30 






.32* 




.50 


11-31 




.22 


.30* 




.76 


11-32 




.22 


.25* 




.55 


11-33 




.55* 






.92 


11-34 


.21 


.22 


.28* 




.73 


11-35 


.32 


.38* 


.23 




.80 


11-36 




.36* 


.31 




.82 


11-37 


.20 


.49* 


.30 




.79 


11-38 




.28 


.45* 




.62 


11-39 




.36 


.45* 




.55 


11-40 






.47* 




..'0 


*Largesc 


loading of Item 











ERIC 



-53- 



Table A-5 

Rotated Factor Loadings 
Germanic Sample 





I 


II 


III 


IV 


Difficulty 


III-l 


— 


.22 


.23* 




.97 


III-2 





.65* 




mmmm 


. 98 


III-3 


.24 


.42* 




, _ 


.83 


III-4 


.25 


.25 


.23 


.50* 


.94 


1 1 1-5 





.32 


.27 


.33* 


.88 


III-6 




.23 




.24* 


.98 


III-7 


.27* 







.24 


.83 


III-8 





.35 


.51* 


— 


.81 


III-9 


.28 


.55* 








.94 


III-IO 


.20 


.21 


• 33* 




.84 


III-ll 





.33* 







.76 


III-12 


.26 


.40* 







.74 


III-13 





..26 


.35* 





.79 


III-14 


.24 




.51* 





.90 


III-15 








.23* 




.37 


III-16 


.29 




.39* 


-.22 


.56 


III-17 








.23* 




,59 


III-18 


.23 


.31 


.24 


-.42* 


.71 


III-19 


.28 


.42* 


.29 




.76 


III-20 





.33* 




_ 


.56 


III-21 


.34 


.49* 


.26 


^ 


.76 


III-22 




.45* 


.26 


^ _ 


.76 


III-23 







. 26* 


_ _ 


.45 


III-24 




.25 


. 39* 





.37 


III-25 





.33* 


.43* 


_ ^ 


.67 


III-26 


__ 




.62* 





.62 


III-27 






.30* 




.43 


III-28 




.27 


.29* 




.66 


III-29 




.34 


.44* 




.56 


III-30 




.27* 




-.20 


.12 


III-31 


.45* 


.42 


.21 


.33 


.97 


III-32 




.39* 






.98 


III-33 


.25 




.20 


.26* 


.83 


III-34 


.36* 


.21 






.97 


III-35 






.51* 




.98 


III-36 


.37* 


.33 


.28 


.36 


.94 


III-37 




.38* 


.31 




.97 


III-38 


.23 


.32 


.42* 




.77 


111-39 




.27 


.28* 


.22 


.94 


III-40 








* 


.90 


III-41 


.32 


.36 


.41* 


.40 


.94 



^Largest loading of item 



(Cont'd.) 



ERIC 



4 • ) 
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Table A-5 

Rotated Factor Loadings 
Germanic Sample 



(Cont'd. ) 





T 


TT 


111 


IV 


Difficulty 


TTT-A9 




.27 


.24 


.38* 


.94 


TT T— A'^ 


2 3* 








.88 


1 1 1"^H'4 




34 


' .50* 


.40 


.90 


1 1 1 J 




.26 


.28* 




.75 


III-46 


.48 


.39 


.50* 


.31 


.97 


III-47 


.32 


.36* 


.36 




QQ 
• OO 


III-48 




.31 


.37* 




.66 


III-49 








20 


.93 


III-50 




.25 


.40* 




.88 


III-51 


.32 


.21 


.36* 


.26 


.81 


III-52 


.33 


.20 


.47* 




.70 


III-53 


.20 


.21 


.36* 


.27 


.85 


III-54 


.38 


.30 


.44 


.49* 


.92 


III-55 


.46 


.28 


.55* 


.20 


.93 


111-56 


.36 


.29 


.48* 


.37 


.93 


111-57 






.54* 




.78 


111-58 




.38 


.55* 


.29 


.85 


111-59 






.71* 


.37 


.89 


111-60 


.36 


.47* 


.42 


.23 


.90 


Variance 


14.08 


15.62 


12.44 


5.34 




Coefficients 






.924 






if Congruence 


.965 


.957 







1 



*Largest loading of item 



ERIC 
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Table A-6 

Rotated Factor Loadings 
Japanese Sample 



I II m IV Difficulty 



I-l 


.51* 











.91 


1-2 


.22* 


_ _ 








.73 


1-3 


.40* 


_ _ 


.20 




.69 


I-A 


.35* 


_ _ 






.56 


1-5 


.28* 


_ _ 


_ _ 





.83 


1-6 


.42* 


.25 


mmmm 





.61 


1-7 


.37* 


.23 


.21 





.61 


1-8 


.51* 










.86 


1-9 


.49* 


.23 


.25 





.72 


I-IO 




.20* 






.37 


I-ll 


.45* 


.35 


^ _ 





.60 


1-12 


.46* 




_ _ 





.54 


1-13 


.42* 


.32 







.74 


I-IA 


.43* 


.29 








.52 


1-15 


.51* 


.29 








.50 


1-16 


* 









.37 


1-17 


.48* 


.23 


_ _ 





.51 


1-18 


.23* 




_ _ 





.33 


1-19 




* 








.56 


1-20 


.21 


.22* 


_ _ 





.35 


1-21 




* 







.83 


1-22 


.52* 


_ _ 





-.27 


.91 




. 36* 


.30 


_ _ 





.86 


I-2A 


.38* 


.22 


_ _ 





.70 


1-25 


.63* 










.77 




S7* 








.88 


1-27 


.59* 


.23 


__ 




.70 


I-?8 


.46* 




_ _ 





.67 


1-29 


.54* 


.21 






.58 


1-30 


.38* 


.29 






.41 


1-31 


.46* 


.21 






.42 


1-32 


.50* 


.31 






0' 

• . A- 


1-33 


.44* 








.50 


1-34 


.47* 








.51 


1-35 


.32* 








.44 


1-36 


.47* 


.22 


.21 




.61 


1-37 




* 






.65 


I-J8 


.37 


.28 


.48* 


-.28 


.85 


1-39 


* 








.22 


1-40 


.41 


.27 


.41* 


-.31 


.83 


1-41 


.32* 




.30 




.65 












(Cont 


^Largest 


loading of Items 
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Table A-6 

Rotated Factor Loadings 
Japanese Sa^le 

(Cont'd,) ^ 

I II m IV Difficulty 



1-42 


.31* 








.84 


1-43 


.57* 


.25 






.6H 


1-44 


.54* 


.23 






.60 


T75 


.47* 


.32 


.39 


-.26 . 


.81 


1-46 


.37* 


.24 






.31 


1-47 


.39* 


.35 






.50 


1-48 


.41* 


.33 






.33 


1-49 


.49* 


.34 






.56 


1-50 


.42* 


.28 






.54 



^Largest loading of items 



1* 



-57- 



Table A-6 

Rotated Factor Loadings 
Japanese Sample 



I II III ly Difficulty 



II-l 


• 32 








70 


II-2 




.36* 


on 




7 


II-3 


. 25 


. 39* 


. J J 


- OA 
. Z4 


Q 7 
. o / 


II-4* 


.27 


.43* 






7 7 


II-5 


.31 


1 o 4 

. 38* 


0 1 




. ou 


II-6 








it 


*\ 7 


II-7 




. 31* 






. D J 


II-8 


. 25 


.26 






1 7 
. 1 / 


II-9 








X 


A 0 

. ^z 


11-10 


.34 


.43* 






. JO 


II^ll 


. 22 


.41* 






A A 
. '♦D 


11-12 


— — 


O T X 

.27* 


Oil ' 
. ZD 




OR 


11-14 


— — 


/ £ X 

.46* 






1 n 
. lo 


11-15 


— — 


.41 


. 45 * 






11-16" 


.27 


m X 

. 39* 


. 3/ 




. Ou 


11-17 


.34* 


. 30 




— 0£. 

. Zo 


R7 
. o / 


11-18 


— 


/ X 

. 34* 






7A 


11-19 


*"— 


O O X 

. 38 * 






7A 


11-20 


— — 


. 30* 


O 1 

. 21 




. 


11-21 


— — 


cox 

. 53* 


. 28 


_ O "7 
. Z / 


7R 
. / o 


11-22 


.27 


/ "7 X 

.47 * 


Oil 

. ZD 




Rn 

. ou 


11-23 




O O X 

.23* 






. u z 


11-24 


o c 

. 25 


. iU* 


0 7 

. z / 






11-25 


.24 


.45* 


.31 


".35 


.79 


11-26 




.35* 


.25 




.62 


11-27 


.22 


.44* 


.35 




.66 


11-28 






.28* 




.75 


11-29 


.22 


.42* 


.23 




. 70 


11-30 




.43* 


.20 




.53 


11-31 




.33* 


.22 




.50 


11-32 




.23* 






.35 


11-33 




■ .43*^ 






.66 


11-34 




.32* 






.48 


11-35 




.40* 






.78 


11-36 








.30* 


.19 


11-37 


.25 


.46* 






.38 


11-38 




.40* 






.44 


11-39 




.27* 






.19 


11-40 




.27 


.31* 




.28 



*Largesc loading of items 



ERIC 
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Table A-6 

Rotated Factor Loadings 
Japanese Sample 



II 



III 



IV 



Difficulty 



I-l 
1-2 
1-3 
1-4 
1-5 

1-7 
1-8 
1-9 
I-IO 



I- 11 



1-12 
1-13 



1-14 
1-15 
1-16 



1=11 



1-18 



1-19 



1-20 



1-21 
1-22 



1-23 
1-24 
1-25 
1-26 



1-27 
1-28 
1-29 
1-30 
1-31 
1-32 
1-33 
1-34 
1-35 
1-36 
1-37 
1-38 
1-39 
1-40 
1-41 



.27 
.25 

.23 

.25 

.30 

,31^ 



,29* 



,24 



.36 
38* 

.25 
39* 
.27 
.25 
.25 



33 



27* 
.33* 

.33 

.26 
.21 



.20 
.21 
.28 

.27 
* 

.32* 



.21 
.31 
.31 

.25 



.42* 

.31* 
.28* 

.28* 
.39 
.32 
.31 

.21* 
.46* 



.37* 
.45* 
.27 
.37* 
.38* 
25* 
.24* 
.36* 
.39* 
.47* 
.34* 
.23 

.44* 
.25* 

.53* 
.31* 

.34* 

.23 

.37* 

.22* 

.37* 

.60* 

.43* 
* 

.48* 
* 

.38 
.30 



.25 

.46* 
.43* 
.38* 

.22 



.20' 



.27* 



.20 



.87 
.85 
.65 
.72 
.63 
.87 
.88 
.73 
.62 
.60 
.72 
.64 
.58 
.40 
.33 
.35 
.57 
.40 
.62 
.45 
.39 
.87 
.25 
.41 
.42 
.27 
.61 
.28 

.26 
* 

.70 
.84 
.41 
.85 
.95 
.69 
.69 
.62 
.88 
.71 
,72 



*Largest loading of items 



(Cont'd. ) 



ERIC 
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Table A'-6 

Rotated Factor Loadings 
Japanese Sample 

(Uut'd.) 





I 


II 


III 


IV 


Difficulty 


IIT-A2 


.25 


. 38* 


. ''3 




. 76 


III-A3 


— 


.52* 






. 33 


III-AA 


— 


. 59* 


. 35 




.60 


III-A5 


.30* 


. 29 


.20 




.52 


III-A6 


.35 


.45 


.54* 


-.28 


.78 


Iil-A7 




.45* 


.32 




.59 


1II-A8 




.35* 


.26 




.58 


III-A9 


.26 


.32 


.34* 


— 


.65 


III-50 


.26 


.41* 


.22 




.52 


III-51 


.28 


.45* 


.37 




.46 


III-52 ' 


.26 


.41* 


.31 


.21 


.36 


III-53 




. 30* 






.50 


III-5A 


.30 


.46* 


.36 


— 


.75 


III-55 


.23 


.56* 


.33 




.70 


III-56 


. 20 


.53* 


.36 




.67 


III-57 




.43* 






.48 


III-58 


.26 


.53* 


.40 




.46 


III-59 




.54* 


.36 




.59 


in-60 


.27 


.49* 


.38 




.7.. 


Variance 


13.12 


14.24 


9.97 


2.59 




Coefficient 












of Congruence 


.983 


.938 


.954 








♦Largest loading of items 



ERIC 
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Table A-7 

Rotated Factor Loadings 
Spanish Sample < 





I 


II 


III 


IV 


Difficulty 


T-1 




26 


.21 




.90 


J. ^ 


-19* 


31 






.66 


T— 1 


AS* 








.79 


t-.A 

X H 


IS* 




. 11 


. 29 


.86 


T-S 


» S2* 




10 


27 


.85 


X \J 


.56* 








.68 


T-7 

X / 




. 36* 


.21 




.75 


T-ft 
X o 


.45* 








.69 


T-Q 

X 7 


.44* 




. 24 




.67 


T-1 n 


SO* 


IR 

• JO 






.61 


i a1 


. U X 








.73 


T-1 9 


. HU 


SI* 






.67 


T-*l 


Al * 

. HX 


. J -3 


21 




.84 


T 1 A 

X— KH 


. HH 


AS* 






. 71 


f- 1 ^ 
X— XD 


SA* 


.44 






.66 


T 1 A 
X— XD 




47* 






.36 


T 17 
x-i / 


S9* 


41 






.61 


T 1 Q 




A7* 






.39 


T~ 1 Q 




19* 






.54 


T on 
1-/U 


9f^ 


. JU 






.46 


T O 1 


7 7* 


11 

. -)X 






.83 


T O O 


A 7* 








. 92 


T O 1 


«;7* 

. 3 J 


91 




21 


.89 


T 9 A 
X— Zw 


A A* 




. 35 




.80 


T 9 
i— / J 


SA* 


9A 






.86 


T 9 A 
i— /O 


SA* 


11 
• JX 


.24 




.85 


T — 9 7 
X— Z / 


Aft* 


9 1 

. X' 






> .85 


1-28 


.41* ^ 




.22 





.77 


1-29 


. 42* 


. 24 






. 73 


1-30 


.64* 


.47 


— 





.66 


T 11 
X— jX 


SI* 


IR 
• JO 






.67 


1-32 


.58* 


.42 






.54 


1-33 


.43 


.51* 






.62 


1-34 


.39* 


.36 






*71 


1-35 


.36 


.45* 






.49 


1-36 


.60* 


.26 • 


.34 




.80 


1-37 




* 






.63 


1-38 


.46* 




.38 




.93 


1-39 


.38* 


.35 


.21 ■ 




.36 


1-40 


.51* 




.36 




.90 


1-41 


.38* 




.36 




.57 


'Largest 


loading of Items 








(Cont'd.) 



ERIC 
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Table A-7 

Rotated Factor Loadings 
Spanish Sample 

(Cont • dO 



I II m IV -Difficulty 



1-42 ' .38* ~ .22 .80 

1-43 .54* ^26 .20 — .70 

1-44 .50* .30 , ~ — .-82 

1-45 .53 *. .29 .25 — .79 

1-46 .27 .39* — ~ .46 

I-4T .33 ."47* .24 — .60 

1-48 ^ .49* v45 — — .56 

1-49 .60* .42 — — .74 

1-50 .52* .33 — ~ .80 



*Largest loading of items 

ERIC 
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Table A-7 

Rotated Factor Loading^ 
Spanish Sample 





I 


T T 

II 


T TT 
ill 


T\7 
iV 




II-l 


. 44* 


1 
• J J 






«9 


II-2 






0 1 * 
. / J* 




7 1 


II-3 


. 42 * 








• CO 


■r T / 

II-4 






9 0* 








. 3^ 


. 44^* 


• ji 


94 


• oO 




O "7 

. 2 / 


. 34* 


in 




ftn 


1 1-7 


• 3V^* 


• j4 






71 


T T Ci 

II-8 


. 2 / 


• Do'* 






11 

• JX 


T T O 






ic 




41 

• ** X 


11-10 


. 44 


Q C 4 


0 n 




A9 


11-11 


. 3/ 


. 22 


17* 




79 


11-12 




• oU ^ 


0 Q 




44 


11-14 




Q 0 + 


• ^O 




. 17 


11-15 




/i 0 ^ 


1 Q 
. J J 




SI 


T T 1 ^ 

II-lo 


. 4U 


1 O 4i 

. 38* 


.2/ 


• HlJ 


Q9 


Ij -17 


. 44 


. 4o * 






ftn 


II- 18 


. 20 * 


. 33 * 


• 30 




fli 

• O J 


11-19 


, 20 


. 34* 


OA. 




Al 
• ox 


11-20 




• 29 




9 A 


• O J 


II- 21 


. 33 


Q Q 4 
. JJ* 






7'^ 


11-22 


. 47 ^ 


.42 


. ^9 




7n 


11-23 


. 25 


. 44* 


o n 
. 2U 




7n 


11-24 


. 26 


. 43* 


• 2o 




* 71 ' 


11-25 




. 34* 


• 32 




• JO ' 


11-26 




49* 


. 42 




A 1 

. oi 


i i — Z / 




44* 


27 




. 70 


11-28 




.49* 


.29 




.80 


11-29 


.28 


.57* 






.72 


11-30 




.45* 






.40 


11-31 




.24 


.26* 




.57 


11-32 






.22* 




.55 


11-33 


.24 


.52* 


.30 




. 74 


11-34 ' 




.35* 






.49 


11-35 




.44* 






.55 


11-36 


.35 


.51* 


.21 




.60 


11-37 


.27 


.60* 


.28 




.-47 


11-38 




.29* 






^ .35 


11-39 




.35* 




.24 


.46 


11-40 




.48* 


.26 




.27 



^Largest loading of items 



(Cont'd. ) 



ERIC 



4 . 



I 
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Table A-7 



Rotated Factor Loadings 
Spanish Sample 



I II , III IV ♦ Difficulty 



III-l 






O 1 4 

• 21* 






III-2 


. 23 


.42* 






Q 1 

. oi 


III-3 


-.24 


.40* 






. 71 


III-4 


.44* 




. 37 




• 00 


1 1 1-5 


.20 


mmmm 


.59* 




o c 

.85 


III-6 






.42* 




.94 


III-7 


.31 


.39* 


.24 




.77 


1 1 1-8 






.50* 




.96 


III-9 


.48 


.50* 


. 20 






III-IO 


— — 


— 


. 32* 




.73 


III-ll 


— 


.33* 


.26 




.70 


III-12 


.35 


.4*4* 


.24 




.59 


III-13 







.44* 




.94 


III-14 


. 24 


.33 " 


.51* 


. 33 


ft o 

. 98 - 


III-15 


— 


.20* 


— 




.44 


III-16 


.22 


.23 


. 38* 




.60 


III-17 


— 


— 


~ — 


* 


.68 


III-18 


.39 


.62* 






.56 


III-19 


— 




.32* 




.84 


I I 1-20 


— "~ 


. 24 


. 30* 




.51 


III-21 


.29 


.59* 






.34 


III-22 


— *" 




.52* 




OA 
. 70 


III-23 


— 


mm 


^ . 41* 




. /u 


I I 1-24 




-.20 


.57* 






III-25 


. 34 


. 24 


. 59* 




on 


III-26 




.21 


:25* 


-.23 


.58 


III-27 






.23* 




.34 


III-28 


.20 


.25 


.27* 




.62 


III-29 


, 27 


.21 


.40* 


-.32 


.50 


III-30 






* 




.20 


III-31 


.29 


.31 


.56* 




.91 


III-32 


.25 




.52* 




.96 


III-33 


.28* 


.21 


,22 




. .57 


III-34 


.28 




.42* 




.90 


III-35 






.47* 




.94 


III-36 


.25 




. 30* 




.74 


III-37 


.31 


.20 


.49* 




.88 


III-38 


.24 


.33 , 


.38* 




.73 


III-39 




.24 


.51* 


.20 


.84 


I I 1-40 






. 30* 




.85 


III-41 


. 35 


.29 


.53* 




.81. 



^Largest loading of Items 



ERIC 



(Cont'd.) 
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Table A- 7 

Rotated Factor Loadings 
Spanish Sample 

(Cont'd.) 

I II m IV Difficulty 



111-42 


. 20 




.53* 





.92 


111-43 






.42* 


.21 


.85 


I I 1-44 


.25 


.31 


.46* 




.76 


III-45 


.22 




.41* 




.66 


III-46 


.31 


.34 


.54* 




.84 


III-47 


.28 


.25 


.48* 




.80 


III-48 


.23 


.26 


.33* 




.70 


III-49 


' .31 


.29 


.51* 


— 


.81 


III-50 






• jj 




. 77 


III-51 


.24 




.57* 


— 


.69 


I I 1-52 


.33 


.36 


.53*. 


— 


.56 


III-53 


.26 


.20 


.44* 




. 7'.' 


III~54 


.44 




.57* 




.86 


III-55 


.43 


.28 


.55* 




.85 


III-56 


.45 


.23 


.46* 




.80 


III-57 




.49* 


.26 




.62 


III-58 


. fi 


.50* 


.37 




.50 


III-59 


.36 


.28 


.46* 


-.27 


.79 


III-60 


.37 


.38 


.42* 


-.21 


.72 


Variance 


18.34 


17.50 


14.63 


2.45 




Coefficient 












of Congruence 


.981 ' 


.923 


.900 







> 



'Largest loading of Items 



ERIC ' 
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Table A-8 

Rotated Factor Loadings 
Total Sample 



I II III IV Difficulty 







23 






.90 


1 it 










.65 


J 




9A 
• ZO 






. 71 


T-A 
1 H 


• H 7 








.74 






• ^ 7 




29 


.85 


T — A 


A7* 








.62 


T_7 

/ 


An* 


•^7 






.76 


i""0 


SA * 


• Zi^ 






.74 


T— Q 


• Z3 


9 7* 
• Z / 




• 


.74 


T in 


AA 


9 A 
• Z^ 






.55% 


T 11 
1-11 




Oft 
• Zo 






63 


T 1 O 

1-lz 




Qft 






.61 


T 11 
1-1 J 


A 1 * 


A 1 
• 41 






.82 


T 1 /. 

I-l^ 


. jy * 


9 9 


9 A 
• Zt* 




63 


T 1 C 
1-1 J 


*;a * 








. 55 


l-lo 










. 30 


T IT 

1-17 


Q 1 4 


9 1 
. Jl 








T 1 Q 

l-lo 


9A * 

• z^ ^ 








.40 


T 1 Q 

1-iy 






• Z J 




. 52 


T on 
1— zu 


9 1 

• zl 


9n 
• zu 


9 ^* 
• ZD'' 




. 39 


1-21 . 


A "7 * 




9 7 
• Z / 




7S 


1-22 




. 3i 




9 A 
• Z4 




T 01 

1-z J 


«;a * 


-~ 






.85 


i— 


AA * 








.65 


i— Z J 


*^9 * 








. 79 


1-26 


.53* 


.33 


.22 


.21 


.81 


1— Z / 


AA* 








. 76 


T-9R 
i Zo 










.69 


1-29 


.53* 


.24 


— 


.23 


.66 


1-30 


.54* 


.30 






.63 


1-31 


.61* 




.22 




.52 


1-32 


.59* 


.27 






.46 


1-33 


.49* 


.31 


.24 




.60 


1-34 


.50* • 


.30 






.63 


1-35 


.39* 


.29 






.53 


1-36 


.49* 




.29 




.68 


1-37 






* 




.56 


1-38 


.46* 


.27 


.34 


.29 


.84 


1-39 


.37 




. 38* 




.28 


1-40 


.39* 


.22 


.32 


.24 


.83 


I-^l 


.29 


.31* 


.27 




.65 












(Cont' 


♦Largest 


loading of item 











ERIC 



♦ 4 
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Table A-8 

Rotated Factor Loadings 
Total Sample 



(Cont'd.) 





I 


II 


III 


IV 


Difficulty 


1-42 


.42* 




.28 




.75 


1-43 


.64* 




.28 




.57 


1-44 


.57*%^ 








.72 


1-45 


.35 


.39* 






.81 


1-46 


.40* 








.38 


1-47 


.47* 




.37 




.45 


1-48 


.58* 


.23 


.25 




.45 


1-49 


.56* 


.40 






.70 


1-50 


.43* 


.30 


.25 




.71 



^Largest loading of item 



-67- 



Table A-8 

Rotated Factor Loadings 
Total Sample 



I II III IV Difficulty 



II-l 


. 37 


. 38* 


.27 


— 


.80 


II-2 


— — 


.27 


. 35* 




.79 


11^3 


. 33 


. 50* 


.28 


.23 


.89 


II-4 


— — 


.39* 


.21 


— 


.82 


II-5 


. 41* 


-36 


. 25 


.20 


.82 


II-6 


. 22 


. 35 


. 23 


— — 


.68 


II-7 


. 20 


. 34* 


.25 


— — 


.69 


II-8 


— — 


.61* 


~— 


.29 


.47 


II-9 


— — 


.24 


.29* 


— 


.55 


11-10 


. 36 


.41 


.47* 


— 


.54 


II-ll 


. 25 


. 31 


.44* 


— 


.60 


11-12 


*~— 


.43* 


.42 


.20 


.45 


11-14 


— — 


.50* 


. 36 


— — 


. 36 


11-15 


— — 


.52* 


.39 


—— 


.52 


11-16 


. 35 


.69* 


— — 


.22 


.91 


11-17 


- 35 


.50* 


— — 


.23 


.87 


11-18 ' 


.27 


.44* 


— — 


— 


.75 


11-19 


.22 


.3L 


.40* 


— 


.65 


11-20 


— 


— 


.33* 


— 


.77 


11-21 


. 22 


.39* 


.26 


.30 


.80 


11-22 


. 30 


.64* 


.21 


— 


.82 


11-23 


. 22 


.33* 


. 30 


— 


.68 


II-2A 


.23 


.37* 


.23 


.20 


.60 


11-25 


.22 


.30 


.34* 


.35 


. 70 


11-26 


.26 


.29 


.52* 


.?7 


.60 


11-27 


.25 


.59 


.27 




.71 


11-28 




.43* 


. 31 




.77 


11-29 


.27 


.52* 


.35 


— 


.77 


11-30 






. 31 




. 56 


11-31 




.33 


,30 




.54 


11-32 






. 30* 




.40 


11-33 


.27 


.55* 


.21 




.76 


11-34 




.32* 


.25 




.56 


11-35 


.25 


.38* 


.27 




.69 


11-36 


.21 


.42* 


.22 




.46 


11-37 


.25 


.30 


. 39* 




.43 


11-38 




.26 


.34* 




.42 


11-39 




.27 


.31* 




.37 


11-40 




.41 


.42* • 




. 38 


*L6rgest 


loading of Item 











ERIC 
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Table A-8 

Rotated Factor Loadings 
Total Samj le 





I 


II 


III 


IV 


Dif f icu 


III-l 


.28 


— 


.39* 


— 


.80 


III-2 


.20 


.55* 


.21 


— 


.87 


1 1 1-3 


— 


.58* 


— 


— 


. 76 


1 1 1-4 


.28 


.36 


.37* 


— 


.70 


III-5 


— 


.37 


.48* 


— 


.66 


III-6 


.37* 


— 


.32 


— 


.79 


III-7 


.24* 


— 


— 


— 


.80 


III-8 


— 


.23 


.46* 


— 


.71 


1 1 1-9 


.42 


.42* 


.28 


— 


.72 


III-IO 


— 


.32 


.35* 


— 


.68 


III-ll 


— 


.26 


. 33* 


— 


.67 


III-12 


.29 


.41* 


— 


— 


.71 


III-13 


— 


.42* 


.42 


.22 


.69 


III-14 


— 


.34 


.54* 


— ^ 


.62 


III-15 


— 


— 


.36* 


— 


.42 


III-16 


— 


.53* 


.36 


— 


.51 


III-17 


— 


— 


.21* 


— 


.54 


III-18 


.26 


.44* 


.40 


— 


.58 


III-19 


.26 


— 


.52* 


— 


.56 


III-20 


— 


.35 


.36 


— 


.54 


III-21 


.22 


.39* 


— 


— 


.47 


III-22 


— 


— 


.52 


— 


.67 


III-23 


— 


.37 


.50* 


.20 


.49 


III-24 


— 


.21 


.62* 


— 


.47 


III-25 


.20 


— 


.60* 


.20 


.49 


III-26 


— 


.42 


.48* 


— 


.53 


III-27 


— 


.33 


.38* 


— 


.48 


III-28 






.63* 




.37 


III-29 




.31 


.53* 




.39 


III-30 




.22 


.34* 


.23 


A 


III-31 


.36 


.35 


.52* 




.75 


III-32 


.31 


.49* 


.20 




.90 


III-33 


.31* 


.27 


.21 




.54 


111-34 


.30* 


.26 


.28 




.83 


III-35 


.42* 


.29 


.24 


.24 


.91 


III-36 


.25 


.20 


.33* 


.24 


.73 


III-37 


.24 


.44* 


.40 




.79 


III-38 


.25 


.37* 


.27 


.24 


.67 


III-39 


.29 




.41* 


.40 


.74 


III-40 






.29 




.74 


III-41 


.39* 


.37 


.38 


.22 


.74 



(Cont'd.) 

^Largest loading of item 

ERIC 



Table A-8 



Rotated Factor Loadings 
Total Saffif^le 

(Cont'd.) 

I II m IV Difficulty 



III-42 


.29 


.28 


.36* 





. 79 


1 1 1-43 


.20 


. 35* 




.34 


. 81 


I I 1-44 


.31 


.33 


.48* 




.68 


III-45 


.30 


.22 


.34* 




.56 


III-46 


.34 


.61* 


.30 


.29 


.81 


III-47 


.31 


.40 


.49* 




69 


III-48 


.24 




.32* 


.26 


.51 


III-49 


. 22 


. 46* 


. 32 




• /D 


T T T 




0 7 
• L 1 


. zy* 




• \jj 


III-51 


.29 


.33 


.43* 


.23 


.58 


I I 1-52 


.29 


.40 


.43* 


.23 


.50 


III-53 




.27 


.34* 




.67 


III-54 


.28 


.36 


.37* 


.21 


.78 


III-55 


.33 


.38 


.45* 


.33 


.75 


III-56 


.28 


.35* 


.47 




.75 


III-57 


.24 


.31 


.37* 




.53 


III-58 


.31 


.41 


.52* 




.55 


III-59 


.38 


.31 


.49* 


.22 


.62 


III-60 


.26 


.41* 


.33 


.22 


.69 


Variance 


16.96 


17.09 


15.28 


3.63 




Coefficients 












of Congruence 


.986 


.972 


.978 







^Largest loading of Item 
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Appcindlx B 
Factors Extension Correlation Matrices 



-72- 



Table B-1 

Factor Extension Matrix for African Sample 



Factor 



Variable I_ 

Sex (1 - Male, 0 - Female) 
Birth year 

Reason for Taking TOEFL: 

Undergraduate Study -.13 
Graduate Study .15 
Other Study 

Professional Licensing 
Employment Requirement 
Other 

Number of Previous TOEFLS 

Studying for Degree 

Listening Comprehension 
(Number Correct) .75 

Structure & Writter. Expression 
(Number Correct) .22 

Reading Comp. & Vocab. 
(Number Correct) .31 



II 



.18 



.57 



.39 



III 

-.17 9 
-.31 

-.23 
.27 



IV 



.2A 



.31 



.50 



-.17 
.18 



.13 



.30 



Note. Loadings less than .10 have been omitLed. 
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Table B-2 

Factor Extension Matrix for Arabic Sample 



Factor 



Variable 



II 



III 



IV 



Sex (1 " Male, 0 - Female) 
>Birth year 

Reason for Taking TOEFL: 
Undergraduate Study 
Graduate Study 
Other Study 
Professional Licensing 
Employment Requirement 
Other 

Number of Previous TOEFLS 

Studying for Degree 

Listening Comprehension 
(Number Correct) 

Structure & Written Expression 
(Number Correct) 

Reading Comp. £> Vocab. 
(Niinber Correct) 



.21 

.14 

-.19 

.10 



.13 



-.15 
.62 
.20 
.21 



.36 



,61 



,43 



-.17 



,15 



.12 



.26 



.35 



.62 



-.27 

-.32 
.22 

-.11 
.20 



Note ; Loadings less than .10 have been omitted. 
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Table B-2 

Factor Extension Matrix for Chinese Saniple 



Factor 



Variable 

Sex (1 » Male, 0 = Female) 
Birth year 

Reason for Taking TOEFL: 
Undergraduate Study 
Graduate Study 
Other Study 

Professional Licensing 
Emplo)Tient Requirement 
Other 

Number of Previous TOEFLS 

Studying for Degree 

Listenihg Comprehension 
(Number Correct) 

Structure & Written Expression 
(Number Correct) 

Reading Comp, & Vocab, 
{, , umber Correct) 



II 



,66 



,25 



.20 



.11 



.35 
.51 
.56 



III 



-•18 



TV 



.18 
.A3 
.49 



-.16 ^ 



Note. LoaGings less than .10 have been omitted. 



/ 



ERIC 



Table B-4 

Factor Extension Matrix for Farsi Sa'hple 



Factor 



Variable 

Sex (1 =« Male, ,0 - Female) 
Birth yeaV 

Reason for Taking TOEFL: 
Undergraduate Study 
Graduate Study 
Other Study 

Professional Licensing 
Employment Requirement 
Other 

Number of Previous TOEFLS 

Studying for Degree 

Listening Comprehension 
(Num?>er Correct) 

Structure & Written Expression 
(Number Correct"^ 

Reading Comp. & Vocab. 
(Number Correct) 



.64 
.33 
.32 



II 

.29 

.27 
-.27' 



III 



IV 



.45 
..49 
.33 



-.21 
.23 



.10 
.43 
.63 



10 
.16 



.17 



Note . Loadings less than .10 have been omitted, 
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Table B-^ 



Factor Extension 


Matrix 


for Germanic 


Sample 












Factor 






Varlble 


I 


II 




III 


IV 


Sex fl ■ Male. 0 " Female; 




.12 




-.10 








.10 


• 


-.15 




t 

Reason for Taking xTOEFL: 












^ Undergraauate Study 




1 1 




— . x^ 






• 






.18 




Other Studv 












Professional Licensing 












Employment Requirement 












Other 












Number of Previous TOEFLS 












Studying for Degree 










.11 


Listening Comprehension 
(Number Correct) 


.58 


.26 




.18 


.17 


Structure & Written Expression 
(Number Corr^^ct) 




.48 




.46 


.18 


Reading Comp. £i Vocab. 
(Number Correct) 


.15 


• .33 




.60 





Note . Loadings less than .10 have been omitted. 
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Table B-6 



Factor Extension 
• 


Matrix for 

• 


Japanese Sample 










Factor 






Variable 


I 


II 


III 


IV 


Sex (1 - Male, 0 = Female) 


.14 




-.27 




Birth year 






-.25 




ReAQon for Taking TOEFL: 










Undergraduate Study 




-.10 


-.21 




Graduate Study 




.lA 


.21 




Other Study 


-.23 


-.12 






Professional Licensing 










Prinl nvm An t" RPfiii 1 ITPlTient 










Other 










Number of Previous TOEFLS 






.10 


.10 


Studying for Degree 






.11 




Listening Comprehension 
(Number Correct) J 


.68 


.35 


.lA 




Structure & Written Expresaion 
(Number Correct) 


.21 


.67 


.28 




Raading Comp. & Vocab. 
(Number Correct) 


.23 


.51 


.53 





Note, i^oadings less than .10 have been ornx'-ted. 
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Table B-7 



Factor Extension 


Matrix for 


Spanish Sample 










Factor 






Variable* 


I 


II 


III 


IV 






• 16 


-.14 


.10 


dxtzq, year 


• 1.J 


99 




.16 


Reason for Taking TOEFL: 


• 








Undergraduate Study 


.11 


.26 


-.26 


.12 


Graduate Study 




1 7 






ucner otuay 










Professional Licensing 










Employment Requirement 










Other 










Number of Previous TOEFLS 










Studying for Degree 




.10 


-.11 




Listening Comprehension 
(Number Correct) 


.62 


.46 






Structure & Written Expression 
(Number Correct) 


.26 


.68 


.23 


-.11 


Reading Comp. & Vocab, 
(Number Correct) 


.32 


.40 


.49 


-.19 

— — w 



Note . Loadings less than .10 l.ave been omitted.. 
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Table B-8 

Factor Extension .Matrix for Total Sample 



Factor 



Variable 


I 


II 


III 


IV 


Sex (1 - Male, 0 = Female) 






-.10 




Birth year 


.15 




-.24 




Reason for Taking TOEFL: 










Undergraduate Study 






-.21 


-.10 


Graduate Study 




- . 10 


.26 


1 7 


Other Study 






-.13 




Professional Licensing 




.11 




-.11 


fanployment Requirement 










At- Vl Al^ 










Number of Previous TOEFLS 








.22 


Studying for Degree 










Listening Comprehension 
(Number Correct) 


.69 


.25 


.28 




Structure & Written Expression 
(Number Correct) 


.23 


.54 


.46 




Reading Comp. L Vocab. 
(Number Correct) 


.26 


.40 


.62 





Note . Loadings less tlan .10 have been omitted. 
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TOEFL 
Research Reports 



Report 1 



The Performance of Native Spedf<ers of English on the Test of English as a Foreign Language Ctark John L D 
November 1977 

Discusses the results of the administration of TOEFL to native speakers of English iust prior to their graduation from a 
coliegp-preparatory high school program Total test score distributions were highly negatively skewed, reinforcing findings of 
earlier studies that TOEFL is not psychometrically appropriate for discnminatmg among native speakers of English v 'th 
respect to English language competence 

An Evaluation of Alternative Iferv Formats for Testing Englishes a Foreign Language Pike. Lewis W Report 2 June 1979 
Descf'bes an extensive research study conducted fror. 1972 to 1974 that was designed !o explore possible changes in the for- 
mat 2nd content of TOfcFL Questions of validation, criterion selection, and content specifications were investigated The 
report includes the results of these findings and discusb3S the implications for TOEFL content specifications and internal 
structure This study contribi *d tc the restructuring of TOEFL beginning in 1976 

The Performance of Non Native Speakers of English on TOEFL and Verbal Aptitude Tests Angelis Paul J Swinton Spencer 
S ar^dCowpi' WiHi^nn R Report 3 October 1979 

Gives the results of a study in whict^400 graduate and undergraduate applicants took TOEFL \ht GRE Verbal or the SAT Ver- 
bal, and the Test of Standard 'Written English (TSWE) Included in the report are comparative data on performance across 
tests and interpretive informatior^ on how combined test results might best be used in the admission process 

An Exploration of Speaking Proficiency Measures m the TOEFL Context Clark John L D 
October 1979 

Describes a three year study involving the development and experimental administration of test formats and item types 
aimed at measuring the English speaking proficiency cf nonnative speakers F-.ctor analysis ana other techniques were used 
to Identify subsets of item formats and individual items having Sdtisfactory correlations wfth the Foreign Service Institute 
criterion interview administered t( e test subjects The results were grouped into a prototype Test of Spoken English 

The Relationship between Scores on the Graduate Management Admission Test and the Test of English as a Foreign 
Language Powers Donald E ReponS December 1980 

Summarizes analyses indicating performance of 6,000 nonnative speakers of English on TOEFL and GMAT In addition to 
comparisons bet//eer> native and nonnative speakers da»a are mck'ded showing performance by language background A 
variety of analyses support the basse differences in the two tests by showing expected GMAT verbal scores for various levels 
of TOEFL scores 

Factor Analysis of the Test of English as a Foreign Language for Several Language Groups Powers Donaid E and Sainton 
Spencer S Report 6 December 1980 

Provides evidence from a set of exploratory analytical techniques that tnree major factors underlie performance on TOEFi. 
Some support is also found ♦or concluding that these factors may be interpreted differently for several language groups The 
report d^-cusses implications ♦or making inferences- based on TOEPI subscores anc^ considerations for future test de 
ve'opmer>t 



and Swinton Spencer S Report 4 



^he Test of Spoken English as a Me-i 
LD and Swinton Sper,( er S Report/' 



i.re of Communicative Ab'lity m English Med'x 
December 1980 



nsfructicnal Settings CtarH John 



ERIC 



Presents the results of a study that examined the performance of foreign teaching assistants on the Tost of Spoken English in 
relation \t> their cl^sroom performance as |udged by students Also inciudes for purposes of comparison data showing per 
formance o^ the same groups of teaching assistants on the Foreign Service oral interview and on TOEFL Based on the 
analyses conducted m the study TSE is shown to be a valid predictor of language abH.t.es for nonr.jtive English speaking^ 
graduate teaching assistants 

tttecfs of ttpm D'sclo^.u e on ^OSn Performance Angehs Pout J Hale Gordon A and ThitKulPau Lawrenc e A Report 8 
Der, ember 198^0 

Reports thp findings of a s'ud- designed to exa.nme the effp ts o* pertormanr p on TOEFL when a subset of items have been 
d snospd prior to an administratK.n Basfd on data from 16 intensive English tram.ng programs the results indicate sigmf. 
cant .ncr^ases m performance m proportion to the number .f items marie available to Mudpnts Details are provided showing 
separate results by language group and by item type 

The above r.>poft^. a^e curf^ntiy available Other r^s.^rch reports a^p planned UitWu-t ,nfuf matmn ntK^ut .iny of the fOEf^L 
Research Repo'>s write to 

TOEFL Program OffK e 
Box 899 

Princeton NJ 08541 USA 



